{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Readme"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Update Apr 26, 2020**\n",
    ">\n",
    "> Ex 1-63 目前已经完成校对。\n",
    "\n",
    "- 题目按照由易到难的顺序排列，最后几题真的比较有挑战性。\n",
    "\n",
    "- 每题我们先给出代码，再给出代码解析。\n",
    "\n",
    "- 为了方便浏览，对于每一题的答案我们只会输出**前5行**。对应在代码中，我们会在所有代码的最后加上`[1:5]`，例如`dt[1:5]`。\n",
    "\n",
    "- 为了充分发挥`data.table`的优点，我们争取所有题目都*只用一行代码解答*。当然这样做会一定程度上导致代码可读性变差，不利于初学者理解。我们在公众号会提供分多行解答的版本。\n",
    "\n",
    "- 如果一题有多个答案，我们会分别列出。\n",
    "\n",
    "- 欢迎大家关注我们的微信公众号**大猫的R语言课堂**。\n",
    "\n",
    "- 如有任何问题请在我们的Github主页开issue。最后感谢renkun！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 15</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>date</th><th scope=col>pre_close</th><th scope=col>open</th><th scope=col>high</th><th scope=col>low</th><th scope=col>close</th><th scope=col>volume</th><th scope=col>amount</th><th scope=col>adj_factor</th><th scope=col>capt</th><th scope=col>index_w50</th><th scope=col>index_w300</th><th scope=col>index_w500</th><th scope=col>industry</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>600000.SH</td><td>20120104</td><td>8.49</td><td>8.54</td><td>8.56</td><td>8.39</td><td>8.41</td><td> 34201379</td><td> 290229551</td><td>6.655275</td><td>125500555680</td><td>0.04640928</td><td>0.02125936</td><td>0</td><td>BANKS</td></tr>\n",
       "\t<tr><td>600000.SH</td><td>20120105</td><td>8.41</td><td>8.47</td><td>8.82</td><td>8.47</td><td>8.65</td><td>132116203</td><td>1144753023</td><td>6.655275</td><td>129082022192</td><td>0.04640928</td><td>0.02125936</td><td>0</td><td>BANKS</td></tr>\n",
       "\t<tr><td>600000.SH</td><td>20120106</td><td>8.65</td><td>8.63</td><td>8.78</td><td>8.62</td><td>8.71</td><td> 61778687</td><td> 537043761</td><td>6.655275</td><td>129977388820</td><td>0.04640928</td><td>0.02125936</td><td>0</td><td>BANKS</td></tr>\n",
       "\t<tr><td>600000.SH</td><td>20120109</td><td>8.71</td><td>8.72</td><td>8.99</td><td>8.68</td><td>8.95</td><td> 80136249</td><td> 711429611</td><td>6.655275</td><td>133558855331</td><td>0.04640928</td><td>0.02125936</td><td>0</td><td>BANKS</td></tr>\n",
       "\t<tr><td>600000.SH</td><td>20120110</td><td>8.95</td><td>8.95</td><td>9.10</td><td>8.88</td><td>9.07</td><td> 72004632</td><td> 647206633</td><td>6.655275</td><td>135349588587</td><td>0.04640928</td><td>0.02125936</td><td>0</td><td>BANKS</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 15\n",
       "\\begin{tabular}{lllllllllllllll}\n",
       " symbol & date & pre\\_close & open & high & low & close & volume & amount & adj\\_factor & capt & index\\_w50 & index\\_w300 & index\\_w500 & industry\\\\\n",
       " <chr> & <int> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <dbl> & <chr>\\\\\n",
       "\\hline\n",
       "\t 600000.SH & 20120104 & 8.49 & 8.54 & 8.56 & 8.39 & 8.41 &  34201379 &  290229551 & 6.655275 & 125500555680 & 0.04640928 & 0.02125936 & 0 & BANKS\\\\\n",
       "\t 600000.SH & 20120105 & 8.41 & 8.47 & 8.82 & 8.47 & 8.65 & 132116203 & 1144753023 & 6.655275 & 129082022192 & 0.04640928 & 0.02125936 & 0 & BANKS\\\\\n",
       "\t 600000.SH & 20120106 & 8.65 & 8.63 & 8.78 & 8.62 & 8.71 &  61778687 &  537043761 & 6.655275 & 129977388820 & 0.04640928 & 0.02125936 & 0 & BANKS\\\\\n",
       "\t 600000.SH & 20120109 & 8.71 & 8.72 & 8.99 & 8.68 & 8.95 &  80136249 &  711429611 & 6.655275 & 133558855331 & 0.04640928 & 0.02125936 & 0 & BANKS\\\\\n",
       "\t 600000.SH & 20120110 & 8.95 & 8.95 & 9.10 & 8.88 & 9.07 &  72004632 &  647206633 & 6.655275 & 135349588587 & 0.04640928 & 0.02125936 & 0 & BANKS\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 15\n",
       "\n",
       "| symbol &lt;chr&gt; | date &lt;int&gt; | pre_close &lt;dbl&gt; | open &lt;dbl&gt; | high &lt;dbl&gt; | low &lt;dbl&gt; | close &lt;dbl&gt; | volume &lt;dbl&gt; | amount &lt;dbl&gt; | adj_factor &lt;dbl&gt; | capt &lt;dbl&gt; | index_w50 &lt;dbl&gt; | index_w300 &lt;dbl&gt; | index_w500 &lt;dbl&gt; | industry &lt;chr&gt; |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 600000.SH | 20120104 | 8.49 | 8.54 | 8.56 | 8.39 | 8.41 |  34201379 |  290229551 | 6.655275 | 125500555680 | 0.04640928 | 0.02125936 | 0 | BANKS |\n",
       "| 600000.SH | 20120105 | 8.41 | 8.47 | 8.82 | 8.47 | 8.65 | 132116203 | 1144753023 | 6.655275 | 129082022192 | 0.04640928 | 0.02125936 | 0 | BANKS |\n",
       "| 600000.SH | 20120106 | 8.65 | 8.63 | 8.78 | 8.62 | 8.71 |  61778687 |  537043761 | 6.655275 | 129977388820 | 0.04640928 | 0.02125936 | 0 | BANKS |\n",
       "| 600000.SH | 20120109 | 8.71 | 8.72 | 8.99 | 8.68 | 8.95 |  80136249 |  711429611 | 6.655275 | 133558855331 | 0.04640928 | 0.02125936 | 0 | BANKS |\n",
       "| 600000.SH | 20120110 | 8.95 | 8.95 | 9.10 | 8.88 | 9.07 |  72004632 |  647206633 | 6.655275 | 135349588587 | 0.04640928 | 0.02125936 | 0 | BANKS |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    date     pre_close open high low  close volume    amount    \n",
       "1 600000.SH 20120104 8.49      8.54 8.56 8.39 8.41   34201379  290229551\n",
       "2 600000.SH 20120105 8.41      8.47 8.82 8.47 8.65  132116203 1144753023\n",
       "3 600000.SH 20120106 8.65      8.63 8.78 8.62 8.71   61778687  537043761\n",
       "4 600000.SH 20120109 8.71      8.72 8.99 8.68 8.95   80136249  711429611\n",
       "5 600000.SH 20120110 8.95      8.95 9.10 8.88 9.07   72004632  647206633\n",
       "  adj_factor capt         index_w50  index_w300 index_w500 industry\n",
       "1 6.655275   125500555680 0.04640928 0.02125936 0          BANKS   \n",
       "2 6.655275   129082022192 0.04640928 0.02125936 0          BANKS   \n",
       "3 6.655275   129977388820 0.04640928 0.02125936 0          BANKS   \n",
       "4 6.655275   133558855331 0.04640928 0.02125936 0          BANKS   \n",
       "5 6.655275   135349588587 0.04640928 0.02125936 0          BANKS   "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# We only need two packages here\n",
    "library(data.table)\n",
    "library(stringr)\n",
    "\n",
    "# set `data_path` to your dir\n",
    "data_path <- \"/Users/raystylee/OneDrive/Code/r-data-practice/data\"\n",
    "setwd(data_path)\n",
    "\n",
    "# read into data\n",
    "data <- readRDS(\"stock-market-data.rds\")\n",
    "data[1:5] # show top 5 obs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 数据集为“面板数据”：包含多个股票（横截面），而每个股票则有多个按照日期排序的变量（时间序列）\n",
    "\n",
    "- 股票代码`symbol` 和日期`date`共同组成了数据集的key，也即每个唯一的`symbol` 和`date`组合决定了一个唯一的观测。\n",
    "\n",
    "- 整个数据集首先按照代码`symbol`排列，其次按照日期`date`排列。\n",
    "\n",
    "- 若干主要变量说明：\n",
    "    - `symbol`：股票代码。.SH 结尾的是沪股，.SZ 结尾的是深股\n",
    "    - `date`：日期\n",
    "    - `pre_close`： 昨收盘\n",
    "    - `open`：开盘价\n",
    "    - `high`：最高价（日内）\n",
    "    - `low`：最低价（日内）\n",
    "    - `close`：收盘价\n",
    "    - `volume`：成交量\n",
    "    - `amount`：成交金额\n",
    "    - `industry`：行业\n",
    "    - `index_w50`：该股票在上证50指数的成分比例\n",
    "    - `index_w300`：该股票在上证300指数的成分比例\n",
    "    - `index_w500`：该股票在中证500指数的成分比例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "# Answer Keys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 哪些股票的代码中包含\"8\"这个数字？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们需要把股票代码`symbol`中包含8的那些观测找出来。我们可以借助`stringr`这个字符串处理包。这一步不难，稍微有些挑战的是去重。如果我们不去重，那么我们会得到非常多的重复观测。例如股票600128，如果它一共有100天的观测，那么我们会出现100个重复结果。为了去重，我们需要借助于`data.table`中的`unique`函数。代码如下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'600008.SH'</li><li>'600018.SH'</li><li>'600028.SH'</li><li>'600038.SH'</li><li>'600048.SH'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item '600008.SH'\n",
       "\\item '600018.SH'\n",
       "\\item '600028.SH'\n",
       "\\item '600038.SH'\n",
       "\\item '600048.SH'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. '600008.SH'\n",
       "2. '600018.SH'\n",
       "3. '600028.SH'\n",
       "4. '600038.SH'\n",
       "5. '600048.SH'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] \"600008.SH\" \"600018.SH\" \"600028.SH\" \"600038.SH\" \"600048.SH\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[str_detect(symbol, \"8\"), unique(symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `str_detect`函数来自`stringr`包，它的输入是一个char vector，输出则是boolean vector，长度与原向量相同。`str_detect(symbol, \"8\")`含义为：对于`symbol`向量，判断其是否含有字符8，如果有，则为True，否则Faulse。\n",
    "\n",
    "- `unique`：找出`symbol`中不重复的值。\n",
    "\n",
    "- 在`data.table`的语法中，**先选择行，再对列进行处理**。所以上述语句会先执行`str_detect`，再执行`unique`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 每天上涨和下跌的股票各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题需要引入分组的概念，并且按照“先分组，后统计”两步走。首先按照题意，我们需要为每个交易日`date`建立一个“组”。其次，对于每个组，我们需要生成两个统计数字：一个统计上涨的个数，一个统计下跌的个数。从下面的结果preview中可以看到，每个`date`都对应了三个观测，一个是“UP”，一个是“DOWN”，一个是“STEADY”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>updown</th><th scope=col>num</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>DOWN  </td><td>2007</td></tr>\n",
       "\t<tr><td>20120104</td><td>STEADY</td><td> 122</td></tr>\n",
       "\t<tr><td>20120104</td><td>UP    </td><td> 191</td></tr>\n",
       "\t<tr><td>20120105</td><td>DOWN  </td><td>2071</td></tr>\n",
       "\t<tr><td>20120105</td><td>STEADY</td><td> 117</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & updown & num\\\\\n",
       " <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & DOWN   & 2007\\\\\n",
       "\t 20120104 & STEADY &  122\\\\\n",
       "\t 20120104 & UP     &  191\\\\\n",
       "\t 20120105 & DOWN   & 2071\\\\\n",
       "\t 20120105 & STEADY &  117\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | updown &lt;chr&gt; | num &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | DOWN   | 2007 |\n",
       "| 20120104 | STEADY |  122 |\n",
       "| 20120104 | UP     |  191 |\n",
       "| 20120105 | DOWN   | 2071 |\n",
       "| 20120105 | STEADY |  117 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     updown num \n",
       "1 20120104 DOWN   2007\n",
       "2 20120104 STEADY  122\n",
       "3 20120104 UP      191\n",
       "4 20120105 DOWN   2071\n",
       "5 20120105 STEADY  117"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, \n",
    "     .(num = uniqueN(symbol)), \n",
    "     keyby = .(date, updown = ifelse(close - pre_close > 0, \"UP\", \n",
    "                              ifelse(close - pre_close < 0, \"DOWN\", \"STEADY\")))\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 代码第一行只有一个逗号。这是因为`data.table`的第一个语句用来对列进行选择，由于我们这里需要对所有列进行统计，所以不需要进行任何操作。\n",
    "\n",
    "- `keyby`用来进行分组，是整个代码的核心。先来看`keyby = .(date, updown)`这个结构，他的意思是，把整个数据集按照`date`和`updown`两个变量进行分组，并依次排序。其中，`updown`是我们新建的字符变量，用来表示分组，它只取两个值：UP, DOWN。这其中的难点是建立`updown`这个变量。我们使用了`ifelse`这个函数。`ifelse(close - pre_close > 0, \"UP\", ifelse(close - pre_close < 0, \"DOWN\", \"STEADY\"))`的意思是，如果今天的收盘价高于昨天的收盘价，那么取值UP，如果收盘价低于昨天的收盘价，取值DOWN，其余的则持平，取值STEADY。\n",
    "\n",
    "- 代码第二行生成了一个新变量`num`。由于在`keyby`语句中我们已经按照日期与涨跌进行了分组，所以这一步我们只需要统计每个组有多少个股票就可以了。我们在这里使用了`uniqueN`这个函数。它是`data.table`内置函数之一，和`unique`几乎执行相同的操作，唯一不同的是，`unique`返回的是不重复的item（是一个向量），而`uniqueN`返回的是不重复的数量（是一个数字）。\n",
    "\n",
    "- 整个代码的执行顺序是：**先选择行（逗号空白行），再分组（`keyby`语句），最后进行组间统计（`num`语句）**。\n",
    "\n",
    "- 我们的答案中，行、列以及分组三条语句各占一行，实际上这仅仅是为了让代码更直观。如果你愿意，`data.table`允许你把所有的代码都写在同一行，就像这样："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[, .(num = uniqueN(symbol)), keyby = .(date, updown = ifelse(close - pre_close > 0, \"UP\", ifelse(close - pre_close < 0, \"DOWN\", \"STEADY\")))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 每天每个交易所上涨、下跌的股票各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这题和`Ex-2`非常类似，唯一的不同就是分组变量多了一个：对于每个交易日，我们不仅需要根据涨跌`updown`分组，还要根据交易所分组。也即，每个交易日都会产生四个subgroup：`SH-DOWN`, `SH-UP`, `SZ-DOWN`, `SZ-UP`.\n",
    "\n",
    "由于股票代码`symbol`的最后两个字符表示交易所（例如，`600123.SH`表示上海交易所，股票代码600123），我们在建立分组变量时需要使用`str_sub`函数截取最后两个字符。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>exchange</th><th scope=col>updown</th><th scope=col>num</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>SH</td><td>DOWN  </td><td> 794</td></tr>\n",
       "\t<tr><td>20120104</td><td>SH</td><td>STEADY</td><td>  42</td></tr>\n",
       "\t<tr><td>20120104</td><td>SH</td><td>UP    </td><td>  85</td></tr>\n",
       "\t<tr><td>20120104</td><td>SZ</td><td>DOWN  </td><td>1213</td></tr>\n",
       "\t<tr><td>20120104</td><td>SZ</td><td>STEADY</td><td>  80</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " date & exchange & updown & num\\\\\n",
       " <int> & <chr> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & SH & DOWN   &  794\\\\\n",
       "\t 20120104 & SH & STEADY &   42\\\\\n",
       "\t 20120104 & SH & UP     &   85\\\\\n",
       "\t 20120104 & SZ & DOWN   & 1213\\\\\n",
       "\t 20120104 & SZ & STEADY &   80\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| date &lt;int&gt; | exchange &lt;chr&gt; | updown &lt;chr&gt; | num &lt;int&gt; |\n",
       "|---|---|---|---|\n",
       "| 20120104 | SH | DOWN   |  794 |\n",
       "| 20120104 | SH | STEADY |   42 |\n",
       "| 20120104 | SH | UP     |   85 |\n",
       "| 20120104 | SZ | DOWN   | 1213 |\n",
       "| 20120104 | SZ | STEADY |   80 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     exchange updown num \n",
       "1 20120104 SH       DOWN    794\n",
       "2 20120104 SH       STEADY   42\n",
       "3 20120104 SH       UP       85\n",
       "4 20120104 SZ       DOWN   1213\n",
       "5 20120104 SZ       STEADY   80"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(num = uniqueN(symbol)), \n",
    "     keyby = .(date, \n",
    "               exchange = str_sub(symbol, start = -2, end = -1), \n",
    "               updown = ifelse(close - pre_close > 0, \"UP\", \n",
    "                        ifelse(close - pre_close < 0, \"DOWN\", \"STEADY\")))\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 在`keyby`语句中，我们创建了三个分组变量，首先是日期`date`，其次是交易所`exchange`（只取SH/SZ两个值），最后是涨跌`updown`。注意这三个变量的先后顺序非常重要，不能颠倒。\n",
    "\n",
    "- 字符串截取函数`str_sub`来自`stringr`包。`str_sub(symbol, start = -2, end = -1)`的意思是截取`symbol`最后两个字符（注意`start`/`end`取了负值）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 沪深300成分股中，每天上涨、下跌的股票各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题仍旧是`Ex-2`的拓展，只不过要求我们进行行选择操作。在`data.table`的`dt[i,j,by]`语法中，`i`代表行选择操作。\n",
    "\n",
    "为了选出沪深300成分股，我们需要用到`index_w300`这个变量。`index_w300`表示一个股票在沪深300指数中的权重，如果大于零，说明它是成分股；如果为零，说明不是成分股。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>updown</th><th scope=col>num</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>DOWN  </td><td>275</td></tr>\n",
       "\t<tr><td>20120104</td><td>STEADY</td><td>  5</td></tr>\n",
       "\t<tr><td>20120104</td><td>UP    </td><td> 20</td></tr>\n",
       "\t<tr><td>20120105</td><td>DOWN  </td><td>242</td></tr>\n",
       "\t<tr><td>20120105</td><td>STEADY</td><td>  8</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & updown & num\\\\\n",
       " <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & DOWN   & 275\\\\\n",
       "\t 20120104 & STEADY &   5\\\\\n",
       "\t 20120104 & UP     &  20\\\\\n",
       "\t 20120105 & DOWN   & 242\\\\\n",
       "\t 20120105 & STEADY &   8\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | updown &lt;chr&gt; | num &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | DOWN   | 275 |\n",
       "| 20120104 | STEADY |   5 |\n",
       "| 20120104 | UP     |  20 |\n",
       "| 20120105 | DOWN   | 242 |\n",
       "| 20120105 | STEADY |   8 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     updown num\n",
       "1 20120104 DOWN   275\n",
       "2 20120104 STEADY   5\n",
       "3 20120104 UP      20\n",
       "4 20120105 DOWN   242\n",
       "5 20120105 STEADY   8"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[index_w300 > 0, \n",
    "     .(num = uniqueN(symbol)), \n",
    "     keyby = .(date, updown = ifelse(close - pre_close > 0, \"UP\", \n",
    "                              ifelse(close - pre_close < 0, \"DOWN\", \"STEADY\")))\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 为了选出沪深300成分股，我们使用了`index_w300 > 0`这个语句。`index_w300`是一个数值变量，与零进行比较运算后会生成一列与原向量等长的**布尔向量**（例如 `c(True, False False, True...)`）。`data.table`只会选择为`True`的那些元素。\n",
    "\n",
    "- 在`data.table`的`dt[i,j,by]`语法中，先执行行选择操作`i`, 再执行分组操作`by`, 最后执行列操作`j`。因此上述代码的执行顺序为：先`index_w300 > 0`，然后`keyby`, 最后生成`num`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. 每天每个行业各有多少只股票？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "仍旧是`Ex-2`的拓展，而且更为简单。我们只需要按照`date`和`industry`进行分组，然后统计每个subgroup的个数即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>stk_num</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>10</td></tr>\n",
       "\t<tr><td>20120104</td><td>AIRLINE</td><td>12</td></tr>\n",
       "\t<tr><td>20120104</td><td>AUTO   </td><td>85</td></tr>\n",
       "\t<tr><td>20120104</td><td>BANKS  </td><td>16</td></tr>\n",
       "\t<tr><td>20120104</td><td>BEV    </td><td>30</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & industry & stk\\_num\\\\\n",
       " <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF & 10\\\\\n",
       "\t 20120104 & AIRLINE & 12\\\\\n",
       "\t 20120104 & AUTO    & 85\\\\\n",
       "\t 20120104 & BANKS   & 16\\\\\n",
       "\t 20120104 & BEV     & 30\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | stk_num &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | AERODEF | 10 |\n",
       "| 20120104 | AIRLINE | 12 |\n",
       "| 20120104 | AUTO    | 85 |\n",
       "| 20120104 | BANKS   | 16 |\n",
       "| 20120104 | BEV     | 30 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry stk_num\n",
       "1 20120104 AERODEF  10     \n",
       "2 20120104 AIRLINE  12     \n",
       "3 20120104 AUTO     85     \n",
       "4 20120104 BANKS    16     \n",
       "5 20120104 BEV      30     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stk_num = uniqueN(symbol)), \n",
    "     keyby = .(date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 我们生成一个新的变量`stk_num`用来表示每天每个行业的股票数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "toc-hr-collapsed": false
   },
   "source": [
    "## 6. 股票数最大的行业和总成交额最大的行业是否总是同一个行业？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>trd_amount</th><th scope=col>stk_num</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td> 493331236</td><td>10</td></tr>\n",
       "\t<tr><td>20120104</td><td>AIRLINE</td><td> 359576492</td><td>12</td></tr>\n",
       "\t<tr><td>20120104</td><td>AUTO   </td><td>2299263905</td><td>85</td></tr>\n",
       "\t<tr><td>20120104</td><td>BANKS  </td><td>3612012715</td><td>16</td></tr>\n",
       "\t<tr><td>20120104</td><td>BEV    </td><td>2946962800</td><td>30</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " date & industry & trd\\_amount & stk\\_num\\\\\n",
       " <int> & <chr> & <dbl> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF &  493331236 & 10\\\\\n",
       "\t 20120104 & AIRLINE &  359576492 & 12\\\\\n",
       "\t 20120104 & AUTO    & 2299263905 & 85\\\\\n",
       "\t 20120104 & BANKS   & 3612012715 & 16\\\\\n",
       "\t 20120104 & BEV     & 2946962800 & 30\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | trd_amount &lt;dbl&gt; | stk_num &lt;int&gt; |\n",
       "|---|---|---|---|\n",
       "| 20120104 | AERODEF |  493331236 | 10 |\n",
       "| 20120104 | AIRLINE |  359576492 | 12 |\n",
       "| 20120104 | AUTO    | 2299263905 | 85 |\n",
       "| 20120104 | BANKS   | 3612012715 | 16 |\n",
       "| 20120104 | BEV     | 2946962800 | 30 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry trd_amount stk_num\n",
       "1 20120104 AERODEF   493331236 10     \n",
       "2 20120104 AIRLINE   359576492 12     \n",
       "3 20120104 AUTO     2299263905 85     \n",
       "4 20120104 BANKS    3612012715 16     \n",
       "5 20120104 BEV      2946962800 30     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data1 <- data[, .(trd_amount = sum(amount), stk_num = uniqueN(symbol)), \n",
    "    keyby = .(date, industry)]\n",
    "data1[1:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result <- data1[data1[, .I[trd_amount == max(trd_amount)], by = date]$V1]$industry == data1[data1[, .I[stk_num == max(stk_num)], by = date]$V1]$industry\n",
    "\n",
    "result[1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>HDWRSEMI</td></tr>\n",
       "\t<tr><td>20120106</td><td>HDWRSEMI</td></tr>\n",
       "\t<tr><td>20120213</td><td>HDWRSEMI</td></tr>\n",
       "\t<tr><td>20120216</td><td>HDWRSEMI</td></tr>\n",
       "\t<tr><td>20120217</td><td>HDWRSEMI</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & industry\\\\\n",
       " <int> & <chr>\\\\\n",
       "\\hline\n",
       "\t 20120104 & HDWRSEMI\\\\\n",
       "\t 20120106 & HDWRSEMI\\\\\n",
       "\t 20120213 & HDWRSEMI\\\\\n",
       "\t 20120216 & HDWRSEMI\\\\\n",
       "\t 20120217 & HDWRSEMI\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; |\n",
       "|---|---|\n",
       "| 20120104 | HDWRSEMI |\n",
       "| 20120106 | HDWRSEMI |\n",
       "| 20120213 | HDWRSEMI |\n",
       "| 20120216 | HDWRSEMI |\n",
       "| 20120217 | HDWRSEMI |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry\n",
       "1 20120104 HDWRSEMI\n",
       "2 20120106 HDWRSEMI\n",
       "3 20120213 HDWRSEMI\n",
       "4 20120216 HDWRSEMI\n",
       "5 20120217 HDWRSEMI"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(trd_amount = sum(amount), stk_num = uniqueN(symbol)), keyby = .(date, industry)\n",
    "    ][, .SD[trd_amount == max(trd_amount) & stk_num == max(stk_num), .(industry)], \n",
    "      keyby = .(date)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. 每天涨幅超过5%、跌幅超过5%的股票各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题的关键思路还是`Ex-2`中的分组。首先，我们自然要对日期分组，然后按照`updown`进行分组。`updown`是用户新建的变量，只取`up5%+`和`down5%+`两个值，一个表示涨幅超过5%，一个表示跌幅超过5%。最后，我们统计每个subgroup的个数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>updown</th><th scope=col>symbol_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>down5%+</td><td>277</td></tr>\n",
       "\t<tr><td>20120104</td><td>up5%+  </td><td> 17</td></tr>\n",
       "\t<tr><td>20120105</td><td>down5%+</td><td>885</td></tr>\n",
       "\t<tr><td>20120105</td><td>up5%+  </td><td> 10</td></tr>\n",
       "\t<tr><td>20120106</td><td>down5%+</td><td> 66</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & updown & symbol\\_amount\\\\\n",
       " <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & down5\\%+ & 277\\\\\n",
       "\t 20120104 & up5\\%+   &  17\\\\\n",
       "\t 20120105 & down5\\%+ & 885\\\\\n",
       "\t 20120105 & up5\\%+   &  10\\\\\n",
       "\t 20120106 & down5\\%+ &  66\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | updown &lt;chr&gt; | symbol_amount &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | down5%+ | 277 |\n",
       "| 20120104 | up5%+   |  17 |\n",
       "| 20120105 | down5%+ | 885 |\n",
       "| 20120105 | up5%+   |  10 |\n",
       "| 20120106 | down5%+ |  66 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     updown  symbol_amount\n",
       "1 20120104 down5%+ 277          \n",
       "2 20120104 up5%+    17          \n",
       "3 20120105 down5%+ 885          \n",
       "4 20120105 up5%+    10          \n",
       "5 20120106 down5%+  66          "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, ':='(ret = (close - pre_close)/pre_close)\n",
    "    ][ret > 0.05 | ret < -0.05, \n",
    "     .(symbol_amount = uniqueN(symbol)), \n",
    "     keyby = .(date, updown = ifelse(ret > 0.05, \"up5%+\", \"down5%+\"))\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 为了方便计算，我们首先在原数据集中新增一个变量`ret`，表示股票的日收益率。`':='(ret = (close - pre_close)/pre_close)`即为新建收益率的语句，其定义为今日收盘价减去昨日收盘价，再除以昨日收盘价\n",
    "\n",
    "- `ret > 0.05 | ret < -0.05`用来删选出收益率超过5%或小于-5%的观测。注意以上运算的结果是一个取值为`True`或`False`的向量，`data.table`最终会挑选出为`True`的那些行。\n",
    "\n",
    "- 我们仍旧使用`ifelse`函数生成`updown`这个变量。`ifelse(ret > 0.05, \"up5%+\", \"down5%+\")`的意思是，如果条件（`ret > 0.05`）成立，那么取值`up5%+`，否则取值`down5%+`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. 每天涨幅前10的股票的总成交额和跌幅前10的股票的总成交额比例是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们需要按照日期进行分组，然后对于每一个交易日，我们需要三个统计值：\n",
    "- `top10`：涨幅前十的股票的总成交额\n",
    "- `bottom10`：跌幅前十的股票的总成交额\n",
    "- `ratio`：`top10/bottom10`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>top10</th><th scope=col>bottom10</th><th scope=col>ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td> 456926638</td><td> 660522056</td><td>0.6917659</td></tr>\n",
       "\t<tr><td>20120105</td><td> 383022701</td><td> 660035625</td><td>0.5803061</td></tr>\n",
       "\t<tr><td>20120106</td><td> 703279194</td><td> 947170130</td><td>0.7425057</td></tr>\n",
       "\t<tr><td>20120109</td><td> 558059840</td><td>1586198246</td><td>0.3518223</td></tr>\n",
       "\t<tr><td>20120110</td><td>2948836472</td><td>1658431604</td><td>1.7780875</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " date & top10 & bottom10 & ratio\\\\\n",
       " <int> & <dbl> & <dbl> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 &  456926638 &  660522056 & 0.6917659\\\\\n",
       "\t 20120105 &  383022701 &  660035625 & 0.5803061\\\\\n",
       "\t 20120106 &  703279194 &  947170130 & 0.7425057\\\\\n",
       "\t 20120109 &  558059840 & 1586198246 & 0.3518223\\\\\n",
       "\t 20120110 & 2948836472 & 1658431604 & 1.7780875\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| date &lt;int&gt; | top10 &lt;dbl&gt; | bottom10 &lt;dbl&gt; | ratio &lt;dbl&gt; |\n",
       "|---|---|---|---|\n",
       "| 20120104 |  456926638 |  660522056 | 0.6917659 |\n",
       "| 20120105 |  383022701 |  660035625 | 0.5803061 |\n",
       "| 20120106 |  703279194 |  947170130 | 0.7425057 |\n",
       "| 20120109 |  558059840 | 1586198246 | 0.3518223 |\n",
       "| 20120110 | 2948836472 | 1658431604 | 1.7780875 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     top10      bottom10   ratio    \n",
       "1 20120104  456926638  660522056 0.6917659\n",
       "2 20120105  383022701  660035625 0.5803061\n",
       "3 20120106  703279194  947170130 0.7425057\n",
       "4 20120109  558059840 1586198246 0.3518223\n",
       "5 20120110 2948836472 1658431604 1.7780875"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, ':='(ret = (close - pre_close)/pre_close)\n",
    "    ][order(date, ret)\n",
    "    ][, .(top10 = sum(amount[1:10]), bottom10 = sum(amount[(.N-9):.N])),\n",
    "      keyby = date\n",
    "    ][, ':='(ratio = top10/bottom10)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 原数据集中，成交量对应变量`amount`。\n",
    "- 第一步，我们为数据集新增一个变量`ret`，表示股票的日收益，对应的语句为`':='(ret = (close - pre_close)/pre_close)`\n",
    "- 其次，我们先按照时间`date`，再按照收益率`ret`从低到高排序，这是为了方便后续计算，对应语句为`order(date, ret)`。其中，`order`是`data.table`中的快速排序函数。\n",
    "- 接着，我们按照日期**分组**，然后计算每一天涨幅前十和跌幅前十的股票的成交额。`amount[1:10]`的意思是取`amount`的前十个观测，`amount[(.N-10):.N]`则表示`amount`的倒数10个观测。由于上一步中我们已经按照收益率`ret`进行了排序，所以`amount[1:10]`就对应了*`ret`最小的10个股票它们对应的成交额*。\n",
    "- `sum`为求和函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. 每天开盘涨停的股票与收盘涨停的股票各有多少？（涨停按照收益率超过1.5%的标准计算）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### key 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还是老样子，首先按照`date`进行分组，然后找出每个交易日中收盘涨停与开盘涨停的股票，最后统计个数。只是题目中把涨停定义为1.5%，似乎和平日里说的10%不一致。不过不影响做题，题目怎么说我们怎么做就行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>n_openlimit</th><th scope=col>n_closelimit</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>325</td><td>  70</td></tr>\n",
       "\t<tr><td>20120105</td><td> 27</td><td>  60</td></tr>\n",
       "\t<tr><td>20120106</td><td> 56</td><td> 743</td></tr>\n",
       "\t<tr><td>20120109</td><td> 73</td><td>2142</td></tr>\n",
       "\t<tr><td>20120110</td><td> 95</td><td>2125</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & n\\_openlimit & n\\_closelimit\\\\\n",
       " <int> & <int> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 325 &   70\\\\\n",
       "\t 20120105 &  27 &   60\\\\\n",
       "\t 20120106 &  56 &  743\\\\\n",
       "\t 20120109 &  73 & 2142\\\\\n",
       "\t 20120110 &  95 & 2125\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | n_openlimit &lt;int&gt; | n_closelimit &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | 325 |   70 |\n",
       "| 20120105 |  27 |   60 |\n",
       "| 20120106 |  56 |  743 |\n",
       "| 20120109 |  73 | 2142 |\n",
       "| 20120110 |  95 | 2125 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     n_openlimit n_closelimit\n",
       "1 20120104 325           70        \n",
       "2 20120105  27           60        \n",
       "3 20120106  56          743        \n",
       "4 20120109  73         2142        \n",
       "5 20120110  95         2125        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, ':='(ret_open = open/pre_close - 1, \n",
    "            ret_close = close/pre_close - 1)\n",
    "    ][, .(n_openlimit = sum(ret_open > 0.015),\n",
    "          n_closelimit = sum(ret_close > 0.015)),\n",
    "      keyby = date\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1-2: 首先我们在原数据集中新增两个变量`ret_open`与`ret_close`，分别表示开盘涨幅与收盘涨幅。\n",
    "- line 3-5: 接着我们按照日期`date`进行分组(`keyby`语句)，然后在每组之内建立两个新的变量`n_openlimit`与`n_closelimit`用来统计开收盘涨停的股票数。这里**比较有技巧的是`sum(ret_open > 0.015)`语句**。首先我们看`ret_open > 0.015`，他的意思是把`ret_open`这个向量的**每一个分量**与1.5%进行比较，如果成立返回`True`，否则`False`。这种比较的最终的结果仍旧是一个向量。`sum(ret_open > 0.015)`则是把这个向量的所有分量相加，要知道`True`在数值计算中被转化成`1`，`False`则转化为`0`，因此`sum(ret_open > 0.015)`最终做的事情是*统计`ret_open`这个向量中有多少个分量是大于0.015的*。\n",
    "- 以上技巧是非常实用的，你也可以自己验证一下，`sum(c(True, False, False))`的结果是不是等于`1`？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key 2 (村长的答案)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>tag</th><th scope=col>date</th><th scope=col>V1</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>closelimit</td><td>20120104</td><td>  70</td></tr>\n",
       "\t<tr><td>closelimit</td><td>20120105</td><td>  60</td></tr>\n",
       "\t<tr><td>closelimit</td><td>20120106</td><td> 743</td></tr>\n",
       "\t<tr><td>closelimit</td><td>20120109</td><td>2142</td></tr>\n",
       "\t<tr><td>closelimit</td><td>20120110</td><td>2125</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " tag & date & V1\\\\\n",
       " <chr> & <int> & <int>\\\\\n",
       "\\hline\n",
       "\t closelimit & 20120104 &   70\\\\\n",
       "\t closelimit & 20120105 &   60\\\\\n",
       "\t closelimit & 20120106 &  743\\\\\n",
       "\t closelimit & 20120109 & 2142\\\\\n",
       "\t closelimit & 20120110 & 2125\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| tag &lt;chr&gt; | date &lt;int&gt; | V1 &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| closelimit | 20120104 |   70 |\n",
       "| closelimit | 20120105 |   60 |\n",
       "| closelimit | 20120106 |  743 |\n",
       "| closelimit | 20120109 | 2142 |\n",
       "| closelimit | 20120110 | 2125 |\n",
       "\n"
      ],
      "text/plain": [
       "  tag        date     V1  \n",
       "1 closelimit 20120104   70\n",
       "2 closelimit 20120105   60\n",
       "3 closelimit 20120106  743\n",
       "4 closelimit 20120109 2142\n",
       "5 closelimit 20120110 2125"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[(close/pre_close - 1) > 0.015 | (open/pre_close - 1) > 0.015, .SD, \n",
    "     by = .(date, symbol, tag = ifelse((close/pre_close - 1) > 0.015, \"closelimit\", \"openlimit\"))\n",
    "    ][, uniqueN(symbol), keyby = .(tag, date)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10. 每天统计最近3天出现过开盘涨停与收盘涨停的股票各有多少只？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了避免歧义，我们首先定义最近三天如下：如果今天是`t`日，那么最近三天则为`[t-3, t-2, t-1]`这三天。当然读者可以有不同理解，这并不影响解题。\n",
    "\n",
    "这题是`Ex-9`的衍生，且具有一定难度，难点在于我们需要进行滚动计算：当处于`t`日时，我们需要取到`n_openlimit[(t-3):(t-1)]`的值（注意`n_openlimit`在`Ex-9`中已经求得，它表示的是每日开收盘涨停的股票数）。我们使用一个非常巧妙与高效的组内循环来解决它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>n_openlimit_3d</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td> NA</td></tr>\n",
       "\t<tr><td>20120105</td><td> NA</td></tr>\n",
       "\t<tr><td>20120106</td><td> NA</td></tr>\n",
       "\t<tr><td>20120109</td><td>408</td></tr>\n",
       "\t<tr><td>20120110</td><td>156</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & n\\_openlimit\\_3d\\\\\n",
       " <int> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120104 &  NA\\\\\n",
       "\t 20120105 &  NA\\\\\n",
       "\t 20120106 &  NA\\\\\n",
       "\t 20120109 & 408\\\\\n",
       "\t 20120110 & 156\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | n_openlimit_3d &lt;int&gt; |\n",
       "|---|---|\n",
       "| 20120104 |  NA |\n",
       "| 20120105 |  NA |\n",
       "| 20120106 |  NA |\n",
       "| 20120109 | 408 |\n",
       "| 20120110 | 156 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     n_openlimit_3d\n",
       "1 20120104  NA           \n",
       "2 20120105  NA           \n",
       "3 20120106  NA           \n",
       "4 20120109 408           \n",
       "5 20120110 156           "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_ex9 <- data[, ':='(ret_open = open/pre_close - 1, \n",
    "            ret_close = close/pre_close - 1)\n",
    "    ][, .(n_openlimit = sum(ret_open > 0.015),\n",
    "          n_closelimit = sum(ret_close > 0.015)),\n",
    "      keyby = date]\n",
    "\n",
    "data_ex9[, \n",
    "     .(date, \n",
    "       n_openlimit_3d = {\n",
    "           l = vector()\n",
    "           for (t in 4:.N) {\n",
    "               l[t] = sum(n_openlimit[(t-3):(t-1)])\n",
    "           }\n",
    "           l\n",
    "       })\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1-9 重复了`Ex-9`的代码，因为我们的计算需要依赖于`Ex-9`的结果\n",
    "- line 8-15 是关键。先不看大括号中的部分，line 8-15的结构是这样的：\n",
    "    ```\n",
    "    .(date, n_openlimit={...})\n",
    "    ```\n",
    "    我们可以看到，这一步的目的是新建两个变量，一个是`date`，它是原数据集就有的，我们直接拿过来；另一个是`n_openlimit_3d`，它是我们在这一步新建的，表示过去三天开盘涨停的股票数。（答案只计算了开盘涨停股票数，但是读者可以自行计算收盘涨停股票数，它们的代码非常相似。）\n",
    "- 我们接着来看`n_openlimit_3d`是如何建立的（大括号中的部分）。首先我们建立一个空的向量（`l = vector()`， line 10），它的每个元素都是`NA`。我们将会**填充**这个空向量`l`，最终让它变成我们的目标向量`n_openlimit_3d`。\n",
    "- line 11-13 是具体的填充过程。因为我们需要向前追溯三天，也就是说，最开头的三天是不会产生有效值的，只有第四天开始才会有值。因此，我们只需要从第四个元素起填充，一直填到最后一个元素，这就是`for (t in 4:.N)`所做的事。其中`.N`是`data.table`自带的特殊变量，表示这个数据集的长度。\n",
    "- line 12 是最关键的一行。首先通过 line 11 的循环，我们得到了一个变量`t`，它表示*当前我们所处的天数*。其次，我们使用`n_openlimit[(t-3):(t-1)]`来获得`[(t-3):(t-1)]`区间的开盘涨停股票数。我们可以看到，`n_openlimit[(t-3):(t-1)]`的结果将是**一个长度为三向量**，每个分量表示某一天的开盘涨停股票数。我们最后使用`sum`对其求和，就得到了过去三天涨停股票的总数。\n",
    "- line 14是为了语法要求而存在的。读者可以把它理解为一个`return`语句：它告诉R，在一系列的操作之后，大括号（line 11-13）最终的输出将会是`l`这个向量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key 2 （村长的答案）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题可能存在一定的陷阱，即是三天内涨停的股票重复计数。例如，`t-1`日涨停的某只股票，很有可能在`t-2`日已经涨停，如果单纯加总每日涨停的股票数量，就会对此股票进行重复计数。因此必须计算三日内涨停的不重复股票数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>tag</th><th scope=col>symbol_amount</th><th scope=col>date</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>closelimit</td><td>  NA</td><td>20120104</td></tr>\n",
       "\t<tr><td>closelimit</td><td>  NA</td><td>20120105</td></tr>\n",
       "\t<tr><td>closelimit</td><td>  NA</td><td>20120106</td></tr>\n",
       "\t<tr><td>closelimit</td><td> 836</td><td>20120109</td></tr>\n",
       "\t<tr><td>closelimit</td><td>2174</td><td>20120110</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " tag & symbol\\_amount & date\\\\\n",
       " <chr> & <int> & <int>\\\\\n",
       "\\hline\n",
       "\t closelimit &   NA & 20120104\\\\\n",
       "\t closelimit &   NA & 20120105\\\\\n",
       "\t closelimit &   NA & 20120106\\\\\n",
       "\t closelimit &  836 & 20120109\\\\\n",
       "\t closelimit & 2174 & 20120110\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| tag &lt;chr&gt; | symbol_amount &lt;int&gt; | date &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| closelimit |   NA | 20120104 |\n",
       "| closelimit |   NA | 20120105 |\n",
       "| closelimit |   NA | 20120106 |\n",
       "| closelimit |  836 | 20120109 |\n",
       "| closelimit | 2174 | 20120110 |\n",
       "\n"
      ],
      "text/plain": [
       "  tag        symbol_amount date    \n",
       "1 closelimit   NA          20120104\n",
       "2 closelimit   NA          20120105\n",
       "3 closelimit   NA          20120106\n",
       "4 closelimit  836          20120109\n",
       "5 closelimit 2174          20120110"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[(close/pre_close - 1) > 0.015 | (open/pre_close - 1) > 0.015, .SD, by = .(date, symbol, tag = ifelse((close/pre_close - 1) > 0.015, \"closelimit\", \"openlimit\"))\n",
    "    ][, .(symbol_list = list(symbol)), keyby = .(tag, date)\n",
    "    ][, .(symbol_amount = {\n",
    "        a <- vector()\n",
    "        for (i in 4:.N) {\n",
    "            a[i] <- lapply(symbol_list[(i-3):(i-1)], unlist) %>% unlist() %>% unique() %>% length()\n",
    "        }\n",
    "        a\n",
    "    }, date), keyby = .(tag)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 选出开盘涨停和收盘涨停的股票，并分别标记为`closelimit`和`openlimit`，生成一个`tag`变量。\n",
    "- line 2 利用`list(symbol)`，外加`keyby = .(tag, date)`，将每天的开盘涨停和收盘涨停的股票代码变为一个单独的list，为`symbol_list`。\n",
    "- line 3 以`tag`进行分组，并对`4:.N`进行循环，将`(i-3):(i-1)`的每一行`symbol_list`，分别去list化，输出为一个无嵌套格式的list：`lapply(symbol_list[(i-3):(i-1)], unlist)`；接着再用`%>% unlist`将这个list，变成一个vector；而后对这个vector求`%>% unique`，去除重复的股票代码；最后求出这个向量的长度`%>% length()`，即三日内开盘涨停和收盘涨停的股票数量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 11. 股票每天的成交额变化率和收益率的相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题关键点主要在于成交额计算，以及其可能出现的数据结果预测和无效数据结果的清洗。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.220968892060145"
      ],
      "text/latex": [
       "0.220968892060145"
      ],
      "text/markdown": [
       "0.220968892060145"
      ],
      "text/plain": [
       "[1] 0.2209689"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(amount_change = (amount - shift(amount, n = 1, fill = NA, type = \"lag\"))/shift(amount, n = 1, type = \"lag\"), \n",
    "         ret = close/pre_close - 1), \n",
    "     by = symbol\n",
    "    ][is.finite(amount_change)\n",
    "    ][, cor(amount_change, ret)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1我们依据成交额变化率计算公式、收益率计算公式，计算出每只股票每天的成交额变化率和收益率。运用`shift(n = 1, fill = NA, type = \"lag\")`函数将所有观测往前滞后一期，而后利用公式`(amount - shift(amount))/shift(amount)`进行成交额计算变化率的计算。\n",
    "- line 2比较关键。首先我们可以设想一下，股票可能发生某一天没有交易量的情况，例如在跳空低开的情况下。那么此时在计算下一个交易日的成交额变化率时，`amount[i-1]`就会为`0`，这时候计算出的`amount_change`就会出现`Inf`这样的值，这在计算相关系数时是不允许的。在这里我们用`is.finite`选择出那些`amount_change`为非无穷大的值，以符合相关系数的计算\n",
    "- line 3则是直接利用`cor`这样一个相关系数计算公式进行计算。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 12. 每天每个行业的总成交额变化率和行业收益率的相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题是`Ex-11`的变形升级版，通过个股的各类数据对行业的总成交额变化率和行业收益率进行计算，在`Ex-11`的基础上增加了对分组的进一步理解，同时加入了每一只股票在行业权重`weight`的计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.394216465079625"
      ],
      "text/latex": [
       "0.394216465079625"
      ],
      "text/markdown": [
       "0.394216465079625"
      ],
      "text/plain": [
       "[1] 0.3942165"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(ind_amount = sum(amount), weight = capt/sum(capt), ret = close/pre_close - 1), keyby = .(date, industry)\n",
    "    ][, .(ind_ret = sum(weight * ret), ind_amount = ind_amount), keyby = .(industry, date)\n",
    "    ][, unique(.SD)\n",
    "    ][, .(ind_amount_change = (ind_amount - shift(ind_amount, n = 1, fill = NA, type = \"lag\"))/shift(ind_amount, n = 1, type = \"lag\"), \n",
    "          ind_ret = ind_ret), \n",
    "      keyby = .(industry)\n",
    "    ][!is.na(ind_amount_change), cor(ind_amount_change, ind_ret)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1以日期`date`和行业`industry`进行分组，而后计算每天每个行业的成交额`ind_amount`，每只股票在行业中的流通市值比重`weight`，以及每只股票每日的收益率`ret`。\n",
    "- line 2以日期`date`和行业`industry`进行分组，计算行业收益率，并继续保留行业成交额`ind_amount`。\n",
    "- line 3在这里做了一个数据的去重，因为我们可以发现，由于我们是精确到个股的数据，按照分组计算出来的行业数据有比较多的重复，因为有很多个股都会对应一个行业，所以在这个地方需要用`unique`进行去重操作。\n",
    "- line 4做法与`Ex-11`类似，以行业`industry`进行分组，分别计算每个行业每天的成交额变化率`ind_amount_change`，这里不再赘述具体做法，可参考`Ex-11`。\n",
    "- line 5计算行业成交额变化率`ind_amount_change`与行业收益率`ind_ret`之间的相关系数。在这里有个特别的处理，虽然交易日中不太可能出现整个行业都没有成交量的情况，但每个行业`ind_amount_change`第一个日期的观察值都为`NA`，需要对所有`NA`值进行删除，因此用了`!is.na(ind_amount_change)`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 13. 每天市场的总成交额变化率和市场收益率相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题相比于`Ex-11`和`Ex-12`又更进了一步，做法与`Ex-12`类似，仅仅是改变了`weight`的计算方式，将股票的`weight`计算为占整个市场的流通市值权重，此题可完全参考`Ex-12`，在此不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.442459215009143"
      ],
      "text/latex": [
       "0.442459215009143"
      ],
      "text/markdown": [
       "0.442459215009143"
      ],
      "text/plain": [
       "[1] 0.4424592"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(mkt_amount = sum(amount), weight = capt/sum(capt), ret = close/pre_close - 1), keyby = date\n",
    "    ][, .(mkt_ret = sum(weight * ret), mkt_amount = mkt_amount), keyby = date\n",
    "    ][, unique(.SD)\n",
    "    ][, .(mkt_amount_change = (mkt_amount - shift(mkt_amount, n = 1, fill = NA, type = \"lag\"))/shift(mkt_amount, n = 1, type = \"lag\"), \n",
    "          mkt_ret = mkt_ret)\n",
    "    ][!is.na(mkt_amount_change), cor(mkt_amount_change, mkt_ret)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 14. 每天市场的总成交额的变化率和所有股票收益率的标准差相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题也是`Ex-11`~`Ex-13`的相似类型，大部分可以参考前三题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "-0.191389346045384"
      ],
      "text/latex": [
       "-0.191389346045384"
      ],
      "text/markdown": [
       "-0.191389346045384"
      ],
      "text/plain": [
       "[1] -0.1913893"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(mkt_amount = sum(amount), ret = close/pre_close - 1, symbol = symbol), keyby = date\n",
    "    ][, .(ret_sd = unique(sd(ret)), mkt_amount = unique(mkt_amount)), keyby = date\n",
    "    ][, .(mkt_amount_change = (mkt_amount - shift(mkt_amount, n = 1, fill = NA, type = \"lag\"))/shift(mkt_amount, n = 1, type = \"lag\"), \n",
    "          ret_sd = ret_sd)\n",
    "    ][!is.na(mkt_amount_change), cor(mkt_amount_change, ret_sd)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 需要注意的是line 2，这里首先计算了所有股票收益率`ret`每一天的标准差`sd(ret)`，而后将line 1计算所得的每一日市场成交总额`mkt_amount`进行合并，而后依据日期进行去重，去重的理由可参考`Ex-12`中的line 3。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 15. 每天每个行业的总成交额变化率和行业内股票收益率的标准差相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题做法与`Ex-14`基本一致，在`Ex-14`基础上加入另一个分组变量`industry`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.104345635868357"
      ],
      "text/latex": [
       "0.104345635868357"
      ],
      "text/markdown": [
       "0.104345635868357"
      ],
      "text/plain": [
       "[1] 0.1043456"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(ind_amount = sum(amount), ret = close/pre_close - 1, symbol = symbol), keyby = .(industry, date)\n",
    "    ][, .(ind_ret_sd = unique(sd(ret)), ind_amount = unique(ind_amount)), keyby = .(industry, date)\n",
    "    ][, .(ind_amount_change = (ind_amount - shift(ind_amount, n = 1, fill = NA, type = \"lag\"))/shift(ind_amount, n = 1, type = \"lag\"),\n",
    "          ind_ret_sd = ind_ret_sd), \n",
    "      keyby = industry\n",
    "    ][!is.na(ind_amount_change), cor(ind_ret_sd, ind_amount_change)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 相比于`Ex-14`，line 1在进行行业总成交额`ind_amount`计算时需要加入`industry`这样一个分组变量，line 2与line 3同理可得。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 16. 上证50、沪深300、中证500指数成分股中，沪股和深股各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题解答上需要感谢frankZhang在村长github主页的提问，提问链接：https://github.com/Ravin515/r-data-practice/issues/1 。这一题的关键在于数据结构的reshape，在这里我们用了`melt`将一个‘宽’的数据集变成了一个‘长’的数据集；此外还在`by`中生成了一列标识出股票属于上证还是深证。为了更加直观展示数据reshape之后的形态，我们将代码分成两部分展示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")\n",
    "    ][value > 0, .(stkcd_amount = uniqueN(symbol)), by = .(index_name, date, type = ifelse(str_detect(symbol, \"SH\"), \"SH\", \"SZ\"))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1中我们首先使用了`melt`，而需要进行处理的变量是`index_w50`、 `index_w300`和`index_w500`，我们需要以这三个变量的变量名为观测值，重新生成一个变量`index_name`，而后这三个变量曾经的观察值生成另外一个变量`value`。接下来，如果要将这三个变量进行批量处理，则需要用到`patterns`这个函数，表示挑选出变量名里面含有“index”字段的变量；之后的`variable.name = \"index_name\"`表明生成一个名为`index_name`的变量，`value`这个变量自动生成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>symbol</th><th scope=col>date</th><th scope=col>pre_close</th><th scope=col>open</th><th scope=col>high</th><th scope=col>low</th><th scope=col>close</th><th scope=col>volume</th><th scope=col>amount</th><th scope=col>adj_factor</th><th scope=col>capt</th><th scope=col>industry</th><th scope=col>index_name</th><th scope=col>value</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>600000.SH   </td><td>20120104    </td><td>8.49        </td><td>8.54        </td><td>8.56        </td><td>8.39        </td><td>8.41        </td><td> 34201379   </td><td> 290229551  </td><td>6.655275    </td><td>125500555680</td><td>BANKS       </td><td>index_w50   </td><td>0.04640928  </td></tr>\n",
       "\t<tr><td>600000.SH   </td><td>20120105    </td><td>8.41        </td><td>8.47        </td><td>8.82        </td><td>8.47        </td><td>8.65        </td><td>132116203   </td><td>1144753023  </td><td>6.655275    </td><td>129082022192</td><td>BANKS       </td><td>index_w50   </td><td>0.04640928  </td></tr>\n",
       "\t<tr><td>600000.SH   </td><td>20120106    </td><td>8.65        </td><td>8.63        </td><td>8.78        </td><td>8.62        </td><td>8.71        </td><td> 61778687   </td><td> 537043761  </td><td>6.655275    </td><td>129977388820</td><td>BANKS       </td><td>index_w50   </td><td>0.04640928  </td></tr>\n",
       "\t<tr><td>600000.SH   </td><td>20120109    </td><td>8.71        </td><td>8.72        </td><td>8.99        </td><td>8.68        </td><td>8.95        </td><td> 80136249   </td><td> 711429611  </td><td>6.655275    </td><td>133558855331</td><td>BANKS       </td><td>index_w50   </td><td>0.04640928  </td></tr>\n",
       "\t<tr><td>600000.SH   </td><td>20120110    </td><td>8.95        </td><td>8.95        </td><td>9.10        </td><td>8.88        </td><td>9.07        </td><td> 72004632   </td><td> 647206633  </td><td>6.655275    </td><td>135349588587</td><td>BANKS       </td><td>index_w50   </td><td>0.04640928  </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllllll}\n",
       " symbol & date & pre\\_close & open & high & low & close & volume & amount & adj\\_factor & capt & industry & index\\_name & value\\\\\n",
       "\\hline\n",
       "\t 600000.SH    & 20120104     & 8.49         & 8.54         & 8.56         & 8.39         & 8.41         &  34201379    &  290229551   & 6.655275     & 125500555680 & BANKS        & index\\_w50  & 0.04640928  \\\\\n",
       "\t 600000.SH    & 20120105     & 8.41         & 8.47         & 8.82         & 8.47         & 8.65         & 132116203    & 1144753023   & 6.655275     & 129082022192 & BANKS        & index\\_w50  & 0.04640928  \\\\\n",
       "\t 600000.SH    & 20120106     & 8.65         & 8.63         & 8.78         & 8.62         & 8.71         &  61778687    &  537043761   & 6.655275     & 129977388820 & BANKS        & index\\_w50  & 0.04640928  \\\\\n",
       "\t 600000.SH    & 20120109     & 8.71         & 8.72         & 8.99         & 8.68         & 8.95         &  80136249    &  711429611   & 6.655275     & 133558855331 & BANKS        & index\\_w50  & 0.04640928  \\\\\n",
       "\t 600000.SH    & 20120110     & 8.95         & 8.95         & 9.10         & 8.88         & 9.07         &  72004632    &  647206633   & 6.655275     & 135349588587 & BANKS        & index\\_w50  & 0.04640928  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| symbol | date | pre_close | open | high | low | close | volume | amount | adj_factor | capt | industry | index_name | value |\n",
       "|---|---|---|---|---|---|---|---|---|---|---|---|---|---|\n",
       "| 600000.SH    | 20120104     | 8.49         | 8.54         | 8.56         | 8.39         | 8.41         |  34201379    |  290229551   | 6.655275     | 125500555680 | BANKS        | index_w50    | 0.04640928   |\n",
       "| 600000.SH    | 20120105     | 8.41         | 8.47         | 8.82         | 8.47         | 8.65         | 132116203    | 1144753023   | 6.655275     | 129082022192 | BANKS        | index_w50    | 0.04640928   |\n",
       "| 600000.SH    | 20120106     | 8.65         | 8.63         | 8.78         | 8.62         | 8.71         |  61778687    |  537043761   | 6.655275     | 129977388820 | BANKS        | index_w50    | 0.04640928   |\n",
       "| 600000.SH    | 20120109     | 8.71         | 8.72         | 8.99         | 8.68         | 8.95         |  80136249    |  711429611   | 6.655275     | 133558855331 | BANKS        | index_w50    | 0.04640928   |\n",
       "| 600000.SH    | 20120110     | 8.95         | 8.95         | 9.10         | 8.88         | 9.07         |  72004632    |  647206633   | 6.655275     | 135349588587 | BANKS        | index_w50    | 0.04640928   |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    date     pre_close open high low  close volume    amount    \n",
       "1 600000.SH 20120104 8.49      8.54 8.56 8.39 8.41   34201379  290229551\n",
       "2 600000.SH 20120105 8.41      8.47 8.82 8.47 8.65  132116203 1144753023\n",
       "3 600000.SH 20120106 8.65      8.63 8.78 8.62 8.71   61778687  537043761\n",
       "4 600000.SH 20120109 8.71      8.72 8.99 8.68 8.95   80136249  711429611\n",
       "5 600000.SH 20120110 8.95      8.95 9.10 8.88 9.07   72004632  647206633\n",
       "  adj_factor capt         industry index_name value     \n",
       "1 6.655275   125500555680 BANKS    index_w50  0.04640928\n",
       "2 6.655275   129082022192 BANKS    index_w50  0.04640928\n",
       "3 6.655275   129977388820 BANKS    index_w50  0.04640928\n",
       "4 6.655275   133558855331 BANKS    index_w50  0.04640928\n",
       "5 6.655275   135349588587 BANKS    index_w50  0.04640928"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 2首先选出了那些`value > 0`的观测行；而后在`by`中用`ifelse`以及`str_detect`生成一个区分沪股和深股的变量`type`，如果其中包含`SH`则生成`SH`, 其他的生成`SZ`；再以`type`，`date`和`index_name`进行分组，计算每个组中股票的数量`stkcd_amount`，这里还使用了去重操作`uniqueN`（注：这里还加入了`date`这个变量进行分组，是因为三大指数的成分股是滚动变化的）。代码和生成结果如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>index_name</th><th scope=col>date</th><th scope=col>type</th><th scope=col>stkcd_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;fct&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>index_w50</td><td>20120104</td><td>SH</td><td>50</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120105</td><td>SH</td><td>50</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120106</td><td>SH</td><td>50</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120109</td><td>SH</td><td>50</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120110</td><td>SH</td><td>50</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " index\\_name & date & type & stkcd\\_amount\\\\\n",
       " <fct> & <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t index\\_w50 & 20120104 & SH & 50\\\\\n",
       "\t index\\_w50 & 20120105 & SH & 50\\\\\n",
       "\t index\\_w50 & 20120106 & SH & 50\\\\\n",
       "\t index\\_w50 & 20120109 & SH & 50\\\\\n",
       "\t index\\_w50 & 20120110 & SH & 50\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| index_name &lt;fct&gt; | date &lt;int&gt; | type &lt;chr&gt; | stkcd_amount &lt;int&gt; |\n",
       "|---|---|---|---|\n",
       "| index_w50 | 20120104 | SH | 50 |\n",
       "| index_w50 | 20120105 | SH | 50 |\n",
       "| index_w50 | 20120106 | SH | 50 |\n",
       "| index_w50 | 20120109 | SH | 50 |\n",
       "| index_w50 | 20120110 | SH | 50 |\n",
       "\n"
      ],
      "text/plain": [
       "  index_name date     type stkcd_amount\n",
       "1 index_w50  20120104 SH   50          \n",
       "2 index_w50  20120105 SH   50          \n",
       "3 index_w50  20120106 SH   50          \n",
       "4 index_w50  20120109 SH   50          \n",
       "5 index_w50  20120110 SH   50          "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")\n",
    "    ][value > 0, .(stkcd_amount = uniqueN(symbol)), by = .(index_name, date, type = ifelse(str_detect(symbol, \"SH\"), \"SH\", \"SZ\"))\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 17. 上证50、沪深300、中证500指数成分股中，行业分布如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题与`Ex-16`类似，只需要在line 2中添加`industry`作为分组变量之一，此题不再赘述。具体数据reshape解析，参考`Ex-16`。。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>index_name</th><th scope=col>date</th><th scope=col>industry</th><th scope=col>stkcd_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;fct&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>index_w50</td><td>20120104</td><td>BANKS</td><td>11</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120105</td><td>BANKS</td><td>11</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120106</td><td>BANKS</td><td>11</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120109</td><td>BANKS</td><td>11</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120110</td><td>BANKS</td><td>11</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " index\\_name & date & industry & stkcd\\_amount\\\\\n",
       " <fct> & <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t index\\_w50 & 20120104 & BANKS & 11\\\\\n",
       "\t index\\_w50 & 20120105 & BANKS & 11\\\\\n",
       "\t index\\_w50 & 20120106 & BANKS & 11\\\\\n",
       "\t index\\_w50 & 20120109 & BANKS & 11\\\\\n",
       "\t index\\_w50 & 20120110 & BANKS & 11\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| index_name &lt;fct&gt; | date &lt;int&gt; | industry &lt;chr&gt; | stkcd_amount &lt;int&gt; |\n",
       "|---|---|---|---|\n",
       "| index_w50 | 20120104 | BANKS | 11 |\n",
       "| index_w50 | 20120105 | BANKS | 11 |\n",
       "| index_w50 | 20120106 | BANKS | 11 |\n",
       "| index_w50 | 20120109 | BANKS | 11 |\n",
       "| index_w50 | 20120110 | BANKS | 11 |\n",
       "\n"
      ],
      "text/plain": [
       "  index_name date     industry stkcd_amount\n",
       "1 index_w50  20120104 BANKS    11          \n",
       "2 index_w50  20120105 BANKS    11          \n",
       "3 index_w50  20120106 BANKS    11          \n",
       "4 index_w50  20120109 BANKS    11          \n",
       "5 index_w50  20120110 BANKS    11          "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")\n",
    "    ][value > 0, .(stkcd_amount = uniqueN(symbol)), by = .(index_name, date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 18. 每天上证50、沪深300、中证500指数成分股的总成交额各是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题与`Ex-16`和`Ex-17`属于同一类型，仅是在line 2以三大指数成分股标识`index_name`和日期`date`分组计算三大指数成分股的总成交额`amount`。具体数据reshape解析，参考`Ex-16`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>index_name</th><th scope=col>date</th><th scope=col>amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;fct&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>index_w50</td><td>20120104</td><td>11372037211</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120105</td><td>15174601949</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120106</td><td>11457969686</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120109</td><td>19783087939</td></tr>\n",
       "\t<tr><td>index_w50</td><td>20120110</td><td>25840613156</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " index\\_name & date & amount\\\\\n",
       " <fct> & <int> & <dbl>\\\\\n",
       "\\hline\n",
       "\t index\\_w50 & 20120104 & 11372037211\\\\\n",
       "\t index\\_w50 & 20120105 & 15174601949\\\\\n",
       "\t index\\_w50 & 20120106 & 11457969686\\\\\n",
       "\t index\\_w50 & 20120109 & 19783087939\\\\\n",
       "\t index\\_w50 & 20120110 & 25840613156\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| index_name &lt;fct&gt; | date &lt;int&gt; | amount &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| index_w50 | 20120104 | 11372037211 |\n",
       "| index_w50 | 20120105 | 15174601949 |\n",
       "| index_w50 | 20120106 | 11457969686 |\n",
       "| index_w50 | 20120109 | 19783087939 |\n",
       "| index_w50 | 20120110 | 25840613156 |\n",
       "\n"
      ],
      "text/plain": [
       "  index_name date     amount     \n",
       "1 index_w50  20120104 11372037211\n",
       "2 index_w50  20120105 15174601949\n",
       "3 index_w50  20120106 11457969686\n",
       "4 index_w50  20120109 19783087939\n",
       "5 index_w50  20120110 25840613156"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")\n",
    "    ][value > 0, .(amount = sum(amount)), by = .(index_name, date)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 19. 上证50、沪深300、中证500指数日收益率的历史波动率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题也是`Ex-16`~`Ex-18`的同一类型，以日期`date`和三大指数成分股的标识`index_name`分组计算历史波动率`vol`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 3 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>index_name</th><th scope=col>vol</th></tr>\n",
       "\t<tr><th scope=col>&lt;fct&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>index_w50 </td><td>0.01234205</td></tr>\n",
       "\t<tr><td>index_w300</td><td>0.01336797</td></tr>\n",
       "\t<tr><td>index_w500</td><td>0.01607531</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 3 × 2\n",
       "\\begin{tabular}{ll}\n",
       " index\\_name & vol\\\\\n",
       " <fct> & <dbl>\\\\\n",
       "\\hline\n",
       "\t index\\_w50  & 0.01234205\\\\\n",
       "\t index\\_w300 & 0.01336797\\\\\n",
       "\t index\\_w500 & 0.01607531\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 3 × 2\n",
       "\n",
       "| index_name &lt;fct&gt; | vol &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| index_w50  | 0.01234205 |\n",
       "| index_w300 | 0.01336797 |\n",
       "| index_w500 | 0.01607531 |\n",
       "\n"
      ],
      "text/plain": [
       "  index_name vol       \n",
       "1 index_w50  0.01234205\n",
       "2 index_w300 0.01336797\n",
       "3 index_w500 0.01607531"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, melt(.SD, measure.vars = patterns(\"index\"), variable.name = \"index_name\")\n",
    "    ][, .(index_ret = sum(value * (close/pre_close - 1))), by = .(date, index_name)\n",
    "    ][, .(vol = sd(index_ret)), by = .(index_name)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1参考`Ex-16`。\n",
    "- line 2以日期`date`和三大指数的成分股的标识`index_name`分组计算每一个指数的历史收益率，其中以三大指数的成分比例`value`为权重，求和每一指数中成分股票的日收益率`close/pre_close-1`。\n",
    "- line 3以`index_name`，分组计算每一个指数的历史波动率`vol`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 20. 上证50、沪深300、中证500指数日收益率的相关系数矩阵？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题主要考察分别计算三大指数的日收益率，并直接生成相关系数矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A matrix: 3 × 3 of type dbl</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>index_w50_ret</th><th scope=col>index_w300_ret</th><th scope=col>index_w500_ret</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>index_w50_ret</th><td>1.0000000</td><td>0.9760849</td><td>0.8476975</td></tr>\n",
       "\t<tr><th scope=row>index_w300_ret</th><td>0.9760849</td><td>1.0000000</td><td>0.9333580</td></tr>\n",
       "\t<tr><th scope=row>index_w500_ret</th><td>0.8476975</td><td>0.9333580</td><td>1.0000000</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A matrix: 3 × 3 of type dbl\n",
       "\\begin{tabular}{r|lll}\n",
       "  & index\\_w50\\_ret & index\\_w300\\_ret & index\\_w500\\_ret\\\\\n",
       "\\hline\n",
       "\tindex\\_w50\\_ret & 1.0000000 & 0.9760849 & 0.8476975\\\\\n",
       "\tindex\\_w300\\_ret & 0.9760849 & 1.0000000 & 0.9333580\\\\\n",
       "\tindex\\_w500\\_ret & 0.8476975 & 0.9333580 & 1.0000000\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A matrix: 3 × 3 of type dbl\n",
       "\n",
       "| <!--/--> | index_w50_ret | index_w300_ret | index_w500_ret |\n",
       "|---|---|---|---|\n",
       "| index_w50_ret | 1.0000000 | 0.9760849 | 0.8476975 |\n",
       "| index_w300_ret | 0.9760849 | 1.0000000 | 0.9333580 |\n",
       "| index_w500_ret | 0.8476975 | 0.9333580 | 1.0000000 |\n",
       "\n"
      ],
      "text/plain": [
       "               index_w50_ret index_w300_ret index_w500_ret\n",
       "index_w50_ret  1.0000000     0.9760849      0.8476975     \n",
       "index_w300_ret 0.9760849     1.0000000      0.9333580     \n",
       "index_w500_ret 0.8476975     0.9333580      1.0000000     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(index_w50_ret = sum(index_w50 * (close/pre_close - 1)), index_w300_ret = sum(index_w300 * (close/pre_close - 1)), index_w500_ret = sum(index_w500 * (close/pre_close - 1))), keyby = date\n",
    "    ][, cor(.SD[, -1])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1以日期`date`分组计算三大指数的日收益率，`index*(close/pre_close - 1)`\n",
    "- line 2去掉变量日期`date`，对其余三个变量求相关系数矩阵。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 21. 上证50、沪深300、去除上证50的沪深300指数日收益率的相关系数矩阵？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题为`Ex-20`的升级版本，主要需要在计算‘去除上证50的沪深300指数收益率’时除去‘上证50’的股票。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A matrix: 3 × 3 of type dbl</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>index_w50_ret</th><th scope=col>index_w300_ret</th><th scope=col>index_w300_50_ret</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>index_w50_ret</th><td>1.0000000</td><td>0.9760849</td><td>0.9294381</td></tr>\n",
       "\t<tr><th scope=row>index_w300_ret</th><td>0.9760849</td><td>1.0000000</td><td>0.9874054</td></tr>\n",
       "\t<tr><th scope=row>index_w300_50_ret</th><td>0.9294381</td><td>0.9874054</td><td>1.0000000</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A matrix: 3 × 3 of type dbl\n",
       "\\begin{tabular}{r|lll}\n",
       "  & index\\_w50\\_ret & index\\_w300\\_ret & index\\_w300\\_50\\_ret\\\\\n",
       "\\hline\n",
       "\tindex\\_w50\\_ret & 1.0000000 & 0.9760849 & 0.9294381\\\\\n",
       "\tindex\\_w300\\_ret & 0.9760849 & 1.0000000 & 0.9874054\\\\\n",
       "\tindex\\_w300\\_50\\_ret & 0.9294381 & 0.9874054 & 1.0000000\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A matrix: 3 × 3 of type dbl\n",
       "\n",
       "| <!--/--> | index_w50_ret | index_w300_ret | index_w300_50_ret |\n",
       "|---|---|---|---|\n",
       "| index_w50_ret | 1.0000000 | 0.9760849 | 0.9294381 |\n",
       "| index_w300_ret | 0.9760849 | 1.0000000 | 0.9874054 |\n",
       "| index_w300_50_ret | 0.9294381 | 0.9874054 | 1.0000000 |\n",
       "\n"
      ],
      "text/plain": [
       "                  index_w50_ret index_w300_ret index_w300_50_ret\n",
       "index_w50_ret     1.0000000     0.9760849      0.9294381        \n",
       "index_w300_ret    0.9760849     1.0000000      0.9874054        \n",
       "index_w300_50_ret 0.9294381     0.9874054      1.0000000        "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(index_w50_ret = sum(index_w50 * (close/pre_close - 1)), index_w300_ret = sum(index_w300 * (close/pre_close - 1)), index_w300_50 = ifelse(index_w50 == 0 & index_w300 > 0, index_w300, 0), close = close, pre_close = pre_close), keyby = date       \n",
    "    ][, .(index_w50_ret = unique(index_w50_ret), index_w300_ret = unique(index_w300_ret), index_w300_50_ret = sum(index_w300_50 * (close/pre_close - 1)) %>% unique()), keyby = date\n",
    "    ][, cor(.SD[, -1])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1在计算出上证50和沪深300的历史收益率的同时，再生成一个变量去除上证50的沪深300指数`index_w300_50`。\n",
    "- line 2将三种指数的收益率都计算出来，而后用`unique`去重。\n",
    "- line 3求相关系数矩阵。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 22. 每天沪深300指数成分占比最大的10只股票是哪些？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题主要考察排序和选择的综合使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>symbol</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>600036.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>600016.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>601318.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>601328.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>601166.SH</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & symbol\\\\\n",
       " <int> & <chr>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 600036.SH\\\\\n",
       "\t 20120104 & 600016.SH\\\\\n",
       "\t 20120104 & 601318.SH\\\\\n",
       "\t 20120104 & 601328.SH\\\\\n",
       "\t 20120104 & 601166.SH\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | symbol &lt;chr&gt; |\n",
       "|---|---|\n",
       "| 20120104 | 600036.SH |\n",
       "| 20120104 | 600016.SH |\n",
       "| 20120104 | 601318.SH |\n",
       "| 20120104 | 601328.SH |\n",
       "| 20120104 | 601166.SH |\n",
       "\n"
      ],
      "text/plain": [
       "  date     symbol   \n",
       "1 20120104 600036.SH\n",
       "2 20120104 600016.SH\n",
       "3 20120104 601318.SH\n",
       "4 20120104 601328.SH\n",
       "5 20120104 601166.SH"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[order(date, -index_w300), .(symbol = symbol[1:10]), by = .(date)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 首先对`date`和`index_w300`分别进行升序和降序排列，也即是先将日期升序排列，而后在每一天中将沪深300的各支股票以成分占比降序排列；接下来利用`by = .(date)`，按照日期进行分组，而后用`.(symbol = symbol[1:10])`选择出成分占比每天排在前十的股票。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 23. 各个行业的平均每日股票数量从大到小排序是什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题主要在于理解题意，并利用排序和分组计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>stkcd_mean</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>HDWRSEMI</td><td>233.3419</td></tr>\n",
       "\t<tr><td>CHEM    </td><td>223.0342</td></tr>\n",
       "\t<tr><td>MACH    </td><td>204.8205</td></tr>\n",
       "\t<tr><td>HEALTH  </td><td>176.7607</td></tr>\n",
       "\t<tr><td>ELECEQP </td><td>134.2393</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " industry & stkcd\\_mean\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t HDWRSEMI & 233.3419\\\\\n",
       "\t CHEM     & 223.0342\\\\\n",
       "\t MACH     & 204.8205\\\\\n",
       "\t HEALTH   & 176.7607\\\\\n",
       "\t ELECEQP  & 134.2393\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| industry &lt;chr&gt; | stkcd_mean &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| HDWRSEMI | 233.3419 |\n",
       "| CHEM     | 223.0342 |\n",
       "| MACH     | 204.8205 |\n",
       "| HEALTH   | 176.7607 |\n",
       "| ELECEQP  | 134.2393 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry stkcd_mean\n",
       "1 HDWRSEMI 233.3419  \n",
       "2 CHEM     223.0342  \n",
       "3 MACH     204.8205  \n",
       "4 HEALTH   176.7607  \n",
       "5 ELECEQP  134.2393  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_amount = uniqueN(symbol)), keyby = .(date, industry)\n",
    "    ][, .(stkcd_mean = mean(stkcd_amount)), by = industry\n",
    "    ][order(-stkcd_mean)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 首先理解题意：计算观测时间内每个行业每天股票的数量，求每个行业股票数量的均值，而后按从大到小排序。\n",
    "- line 1 利用分组计算出了每个行业每天的股票数量`stkcd_amount`，line 2 以行业`industry`进行分组计算每个行业的股票数量的均值，line 3则依据`stkcd_mean`进行降序排列。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 24. 每个行业每天成交额最大的一只股票代码是什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题与`Ex-22`类似，这里不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>symbol</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>000768.SZ</td></tr>\n",
       "\t<tr><td>20120104</td><td>AIRLINE</td><td>600029.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>AUTO   </td><td>600104.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>BANKS  </td><td>600036.SH</td></tr>\n",
       "\t<tr><td>20120104</td><td>BEV    </td><td>600519.SH</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & industry & symbol\\\\\n",
       " <int> & <chr> & <chr>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF & 000768.SZ\\\\\n",
       "\t 20120104 & AIRLINE & 600029.SH\\\\\n",
       "\t 20120104 & AUTO    & 600104.SH\\\\\n",
       "\t 20120104 & BANKS   & 600036.SH\\\\\n",
       "\t 20120104 & BEV     & 600519.SH\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | symbol &lt;chr&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | AERODEF | 000768.SZ |\n",
       "| 20120104 | AIRLINE | 600029.SH |\n",
       "| 20120104 | AUTO    | 600104.SH |\n",
       "| 20120104 | BANKS   | 600036.SH |\n",
       "| 20120104 | BEV     | 600519.SH |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry symbol   \n",
       "1 20120104 AERODEF  000768.SZ\n",
       "2 20120104 AIRLINE  600029.SH\n",
       "3 20120104 AUTO     600104.SH\n",
       "4 20120104 BANKS    600036.SH\n",
       "5 20120104 BEV      600519.SH"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[order(-amount), .(symbol = symbol[1]), keyby = .(date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 25. 每个行业每天最大成交额是最小成交额的几倍？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题主要也是考察排序后的分组计算，包括理解题意。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>times</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td> 12.81924</td></tr>\n",
       "\t<tr><td>20120104</td><td>AIRLINE</td><td> 47.38407</td></tr>\n",
       "\t<tr><td>20120104</td><td>AUTO   </td><td>304.90510</td></tr>\n",
       "\t<tr><td>20120104</td><td>BANKS  </td><td> 11.62224</td></tr>\n",
       "\t<tr><td>20120104</td><td>BEV    </td><td>191.00700</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & industry & times\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF &  12.81924\\\\\n",
       "\t 20120104 & AIRLINE &  47.38407\\\\\n",
       "\t 20120104 & AUTO    & 304.90510\\\\\n",
       "\t 20120104 & BANKS   &  11.62224\\\\\n",
       "\t 20120104 & BEV     & 191.00700\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | times &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | AERODEF |  12.81924 |\n",
       "| 20120104 | AIRLINE |  47.38407 |\n",
       "| 20120104 | AUTO    | 304.90510 |\n",
       "| 20120104 | BANKS   |  11.62224 |\n",
       "| 20120104 | BEV     | 191.00700 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry times    \n",
       "1 20120104 AERODEF   12.81924\n",
       "2 20120104 AIRLINE   47.38407\n",
       "3 20120104 AUTO     304.90510\n",
       "4 20120104 BANKS     11.62224\n",
       "5 20120104 BEV      191.00700"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[order(date, industry, -amount)\n",
    "    ][amount > 0, .(times = amount[1]/amount[.N]), by = .(date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 本题根据`Ex-24`题意，推测本题含义应是：行业每天最大成交额的股票是最小成交股票的几倍。\n",
    "- 首先依据日期`date`、行业`industry`和成交额`amount`分别进行升序，升序和降序排列，并且删除所有成交额`amount`等于0的观测，接下来以日期`date`和行业`industry`进行分组，最后在每组中以`amount`最大值除以`amount`最小值：`times = amount[1]/amount[.N]`。\n",
    "- 注：在这里`i`中的排序和选择的操作的代码分成了两步，这是因为这两个部分不能够以`order(date, industry, -amount) & amount > 0`形式书写，这样书写其中的排序部分会失效。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 26. 每个行业每天成交额最大的5只股票和成交额总和是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题与`Ex-25`类似，总体可参考`Ex-25`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>symbol</th><th scope=col>amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>000768.SZ</td><td>384758246</td></tr>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>600316.SH</td><td>384758246</td></tr>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>002025.SZ</td><td>384758246</td></tr>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>600893.SH</td><td>384758246</td></tr>\n",
       "\t<tr><td>20120104</td><td>AERODEF</td><td>600118.SH</td><td>384758246</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " date & industry & symbol & amount\\\\\n",
       " <int> & <chr> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF & 000768.SZ & 384758246\\\\\n",
       "\t 20120104 & AERODEF & 600316.SH & 384758246\\\\\n",
       "\t 20120104 & AERODEF & 002025.SZ & 384758246\\\\\n",
       "\t 20120104 & AERODEF & 600893.SH & 384758246\\\\\n",
       "\t 20120104 & AERODEF & 600118.SH & 384758246\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | symbol &lt;chr&gt; | amount &lt;dbl&gt; |\n",
       "|---|---|---|---|\n",
       "| 20120104 | AERODEF | 000768.SZ | 384758246 |\n",
       "| 20120104 | AERODEF | 600316.SH | 384758246 |\n",
       "| 20120104 | AERODEF | 002025.SZ | 384758246 |\n",
       "| 20120104 | AERODEF | 600893.SH | 384758246 |\n",
       "| 20120104 | AERODEF | 600118.SH | 384758246 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry symbol    amount   \n",
       "1 20120104 AERODEF  000768.SZ 384758246\n",
       "2 20120104 AERODEF  600316.SH 384758246\n",
       "3 20120104 AERODEF  002025.SZ 384758246\n",
       "4 20120104 AERODEF  600893.SH 384758246\n",
       "5 20120104 AERODEF  600118.SH 384758246"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[order(date, industry, -amount), .(symbol = symbol[1:5], amount = sum(amount[1:5])),\n",
    "     by = .(date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 27. 每个行业每天成交额超过该行业中股票成交额80%分位数的股票的平均收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题的关键在于运用了分组以后的`.SD`选择。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th><th scope=col>aver_ret</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>BANKS</td><td>-0.007846674</td></tr>\n",
       "\t<tr><td>20120105</td><td>BANKS</td><td> 0.021967669</td></tr>\n",
       "\t<tr><td>20120106</td><td>BANKS</td><td> 0.007687184</td></tr>\n",
       "\t<tr><td>20120109</td><td>BANKS</td><td> 0.025649750</td></tr>\n",
       "\t<tr><td>20120110</td><td>BANKS</td><td> 0.010355259</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & industry & aver\\_ret\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & BANKS & -0.007846674\\\\\n",
       "\t 20120105 & BANKS &  0.021967669\\\\\n",
       "\t 20120106 & BANKS &  0.007687184\\\\\n",
       "\t 20120109 & BANKS &  0.025649750\\\\\n",
       "\t 20120110 & BANKS &  0.010355259\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; | aver_ret &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | BANKS | -0.007846674 |\n",
       "| 20120105 | BANKS |  0.021967669 |\n",
       "| 20120106 | BANKS |  0.007687184 |\n",
       "| 20120109 | BANKS |  0.025649750 |\n",
       "| 20120110 | BANKS |  0.010355259 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry aver_ret    \n",
       "1 20120104 BANKS    -0.007846674\n",
       "2 20120105 BANKS     0.021967669\n",
       "3 20120106 BANKS     0.007687184\n",
       "4 20120109 BANKS     0.025649750\n",
       "5 20120110 BANKS     0.010355259"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(symbol = symbol, ret = close/pre_close - 1, amount = amount, industry, date)\n",
    "    ][, .SD[amount > quantile(amount, 0.8)], by = .(date, industry)\n",
    "    ][, .(aver_ret = mean(ret)), by = .(date, industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算了个股每日的收益率`ret`，再将其余需要的变量挑选出来。\n",
    "- line 2 是本题的关键。首先根据`date`和`industry`进行分组，而后在分组的`.SD`中选择每天成交额超过该行业中股票成交额80%分位数的股票：`.SD[amount > quantile(amount, 0.8)]`，这样就将每日每个行业中超过本行业80%分位数的股票选择出来了。\n",
    "- line 3 则是分组计算每日每行业这些股票的平均收益率`aver_ret`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 28. 每天成交额最大的10%的股票的平均收益率和成交额最小的10%的股票的平均收益率的相关系数是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题主要运用了`dcast`将一个‘长’的表变成一个‘宽’的表，还有关于R中变量名引用问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.88838742129573"
      ],
      "text/latex": [
       "0.88838742129573"
      ],
      "text/markdown": [
       "0.88838742129573"
      ],
      "text/plain": [
       "[1] 0.8883874"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(symbol = symbol, ret = close/pre_close - 1, amount = amount, date)\n",
    "    ][, .(tag = ifelse(amount > quantile(amount, 0.9), \"max10%\", ifelse(amount < quantile(amount, 0.1), \"min10%\", \"others\")), ret), by = .(date)\n",
    "    ][, .(ret_aver = mean(ret)), by = .(date, tag)\n",
    "    ][tag != \"others\", dcast(.SD, date ~ tag, value.var = \"ret_aver\")\n",
    "    ][, cor(`max10%`, `min10%`)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 与前一题类似计算出个股收益率`ret`，而后挑选出需要的变量。\n",
    "- line 2 生成一个变量`tag`，s首先以日期`date`进行分组，而后以`quantile(amount, 0.9)`和`quantile(amount, 0.1)`为两个标识，生成三个观测值`max10%`、`min10%`和`others`。\n",
    "- line 3 接下来根据日期`date`和标识`tag`，计算三个分组在每日的平均收益率`ret_aver`。\n",
    "- line 4 在删除`tag = \"other\"`的这些观测之后，用`dcast`将表进行变形，把观测值`max10%`和`min10%`变成两个变量名，而后在这两个变量名下填充`ret_aver`的观测值。\n",
    "- line 5 计算`max10%`和`min10%`这两个变量的相关系数。因为在变量名中出现的`%`，会在函数中自动识别为函数`%`，如果需要讲变量名进行引用，则需要运用引用符号``` `` ```这个函数，"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 29. 每天哪些行业的平均成交额高于全市场平均成交额？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这一题主要是分组计算行业平均成交额`amount_ind_aver`和市场平均成交额`amount_mkt_aver`，而后进行比较。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>industry</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>AERODEF </td></tr>\n",
       "\t<tr><td>20120104</td><td>BANKS   </td></tr>\n",
       "\t<tr><td>20120104</td><td>BEV     </td></tr>\n",
       "\t<tr><td>20120104</td><td>CONMAT  </td></tr>\n",
       "\t<tr><td>20120104</td><td>DVFININS</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & industry\\\\\n",
       " <int> & <chr>\\\\\n",
       "\\hline\n",
       "\t 20120104 & AERODEF \\\\\n",
       "\t 20120104 & BANKS   \\\\\n",
       "\t 20120104 & BEV     \\\\\n",
       "\t 20120104 & CONMAT  \\\\\n",
       "\t 20120104 & DVFININS\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | industry &lt;chr&gt; |\n",
       "|---|---|\n",
       "| 20120104 | AERODEF  |\n",
       "| 20120104 | BANKS    |\n",
       "| 20120104 | BEV      |\n",
       "| 20120104 | CONMAT   |\n",
       "| 20120104 | DVFININS |\n",
       "\n"
      ],
      "text/plain": [
       "  date     industry\n",
       "1 20120104 AERODEF \n",
       "2 20120104 BANKS   \n",
       "3 20120104 BEV     \n",
       "4 20120104 CONMAT  \n",
       "5 20120104 DVFININS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(amount_mkt_aver = mean(amount), industry = industry, amount = amount), keyby = .(date)\n",
    "    ][, .(amount_ind_aver = mean(amount), amount_mkt_aver), keyby = .(date, industry)\n",
    "    ][amount_ind_aver > amount_mkt_aver, unique(.SD[, 1:2])\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 分组计算每天的市场平均成交额`amount_mkt_aver`，而后选择出需要的变量。\n",
    "- line 2 分组计算每天每个行业的平均成交额`amount_ind_aver`。\n",
    "- line 3 将每天每个行业平均成交额`amount_ind_aver`高于市场平均成交额`amount_mkt_aver`的观测行选择出来，而后只保留日期`date`和行业`industry`这两个变量，由于这时的数据是以个股为单个观测值的数据，所以最后对数据进行去重。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 30. 每天每个股票对市场的超额收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于利用公式$R_{it} = \\alpha_i + \\beta_iR^m_{it} + \\epsilon_{it}$，其中$R_{it}$为个股每日的收益率,$R^m_{it}$为市场每日的收益率, 而后进行回归，提取回归系数$\\alpha_i$和$\\beta_i$，最后依据公式$AR_{it} = R_{it} - \\overline R_{it}$进行超额收益率计算，其中$\\overline R_{it} = \\overline \\alpha_{i} + \\overline \\beta_{i} R^m_{it}$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>symbol</th><th scope=col>abnr_ret</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>000001.SZ</td><td>-0.015698359</td></tr>\n",
       "\t<tr><td>20120104</td><td>000002.SZ</td><td>-0.003998231</td></tr>\n",
       "\t<tr><td>20120104</td><td>000004.SZ</td><td> 0.001884933</td></tr>\n",
       "\t<tr><td>20120104</td><td>000005.SZ</td><td> 0.002478519</td></tr>\n",
       "\t<tr><td>20120104</td><td>000006.SZ</td><td> 0.006396669</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & symbol & abnr\\_ret\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 000001.SZ & -0.015698359\\\\\n",
       "\t 20120104 & 000002.SZ & -0.003998231\\\\\n",
       "\t 20120104 & 000004.SZ &  0.001884933\\\\\n",
       "\t 20120104 & 000005.SZ &  0.002478519\\\\\n",
       "\t 20120104 & 000006.SZ &  0.006396669\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | symbol &lt;chr&gt; | abnr_ret &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | 000001.SZ | -0.015698359 |\n",
       "| 20120104 | 000002.SZ | -0.003998231 |\n",
       "| 20120104 | 000004.SZ |  0.001884933 |\n",
       "| 20120104 | 000005.SZ |  0.002478519 |\n",
       "| 20120104 | 000006.SZ |  0.006396669 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     symbol    abnr_ret    \n",
       "1 20120104 000001.SZ -0.015698359\n",
       "2 20120104 000002.SZ -0.003998231\n",
       "3 20120104 000004.SZ  0.001884933\n",
       "4 20120104 000005.SZ  0.002478519\n",
       "5 20120104 000006.SZ  0.006396669"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, weight = capt/sum(capt), symbol), keyby = date\n",
    "    ][, .(mkt_ret = sum(weight*stkcd_ret), stkcd_ret, symbol), keyby = date\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ mkt_ret))[1], beta = coef(lm(stkcd_ret ~ mkt_ret))[2], mkt_ret, stkcd_ret, date), by = symbol\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * mkt_ret), keyby = .(date, symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 首先计算每日的个股收益率`stkcd_ret`以及市场总每只股票流通市值占比的权重`weight`。\n",
    "- line 2 计算每日的市场收益率`mkt_ret`。\n",
    "- line 3 依据公式 $R_{it} = \\alpha_i + \\beta_iR^m_{it} + \\epsilon_{it}$，利用回归模型`lm(stkcd_ret ~ mkt_ret)`分别计算出回归的截距项`alpha`和`beta`。\n",
    "- line 4 依据公式 $AR_{it} = R_{it} - \\overline R_{it}$ 进行每只股票每天的超额收益率`abnr_net`计算。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 31. 每天每个股票对市场去除自身的超额收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于市场除去每只股票自身的超额收益率的计算，公式如下：$ ret^m_i = \\sum_{i = 1, i \\ne k}^{n-1} \\frac {capt_i \\cdot ret_i}{\\sum_{j = 1, j \\ne k}^{n-1} capt_j} $。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>symbol</th><th scope=col>abnr_ret</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>000001.SZ</td><td>-0.015768934</td></tr>\n",
       "\t<tr><td>20120104</td><td>000002.SZ</td><td>-0.004096192</td></tr>\n",
       "\t<tr><td>20120104</td><td>000004.SZ</td><td> 0.001883449</td></tr>\n",
       "\t<tr><td>20120104</td><td>000005.SZ</td><td> 0.002473336</td></tr>\n",
       "\t<tr><td>20120104</td><td>000006.SZ</td><td> 0.006376279</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & symbol & abnr\\_ret\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 000001.SZ & -0.015768934\\\\\n",
       "\t 20120104 & 000002.SZ & -0.004096192\\\\\n",
       "\t 20120104 & 000004.SZ &  0.001883449\\\\\n",
       "\t 20120104 & 000005.SZ &  0.002473336\\\\\n",
       "\t 20120104 & 000006.SZ &  0.006376279\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | symbol &lt;chr&gt; | abnr_ret &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | 000001.SZ | -0.015768934 |\n",
       "| 20120104 | 000002.SZ | -0.004096192 |\n",
       "| 20120104 | 000004.SZ |  0.001883449 |\n",
       "| 20120104 | 000005.SZ |  0.002473336 |\n",
       "| 20120104 | 000006.SZ |  0.006376279 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     symbol    abnr_ret    \n",
       "1 20120104 000001.SZ -0.015768934\n",
       "2 20120104 000002.SZ -0.004096192\n",
       "3 20120104 000004.SZ  0.001883449\n",
       "4 20120104 000005.SZ  0.002473336\n",
       "5 20120104 000006.SZ  0.006376279"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, mkt_capt_outself =  sum(capt) - capt, capt), keyby = date\n",
    "    ][, .(mkt_ret_outself = {\n",
    "        a <- vector()\n",
    "        for (i in 1:.N) {\n",
    "            a[i] <- sum((capt[-i]/mkt_capt_outself[i]) * stkcd_ret[-i])\n",
    "        }\n",
    "        a\n",
    "    }, stkcd_ret, symbol), \n",
    "      keyby = date\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ mkt_ret_outself))[1], beta = coef(lm(stkcd_ret ~ mkt_ret_outself))[2], mkt_ret_outself, stkcd_ret, date), by = symbol\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * mkt_ret_outself), keyby = .(date, symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票每日的收益率`stkcd_ret`，并且计算出市场中除这只股票自己本身之外其他股票的总流通市值`mkt_capt_outself`。\n",
    "- line 2 这一步为此题的关键，我们以日期`date`分组，而后遍历所有的股票，将所有其他股票当天的流通市值`capt[-i]`，除以除去自身流通市值的市场整体流通市值`mkt_capt_outself[i]`，即`capt[-i]/mkt_capt_outself[i]`。而后利用R当中两个向量相乘的原理，乘以这些股票当天的收益率`stkcd_ret[-i]`，最后整体加和得出市场除去每只股票自身的收益率：`sum((capt[-i]/mkt_capt_outself[i]) * stkcd_ret[-i])`。\n",
    "- line 3 依据公式 $R_{it} = \\alpha_i + \\beta_iR^m_{it} + \\epsilon_{it}$，利用回归模型`lm(stkcd_ret ~ mkt_ret_outself)`分别计算出回归的截距项`alpha`和`beta`。\n",
    "- line 4 依据公式 $AR_{it} = R_{it} - \\overline R_{it}$ 进行每只股票每天的超额收益率`abnr_net`计算。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 32. 每天每个股票对行业的超额收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与`Ex-30`类似，可完全参考`Ex-30`，此处不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>symbol</th><th scope=col>abnr_ret</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>000001.SZ</td><td>-0.0187202985</td></tr>\n",
       "\t<tr><td>20120104</td><td>000002.SZ</td><td>-0.0008383009</td></tr>\n",
       "\t<tr><td>20120104</td><td>000004.SZ</td><td> 0.0114847517</td></tr>\n",
       "\t<tr><td>20120104</td><td>000005.SZ</td><td> 0.0047238201</td></tr>\n",
       "\t<tr><td>20120104</td><td>000006.SZ</td><td> 0.0108104722</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & symbol & abnr\\_ret\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 000001.SZ & -0.0187202985\\\\\n",
       "\t 20120104 & 000002.SZ & -0.0008383009\\\\\n",
       "\t 20120104 & 000004.SZ &  0.0114847517\\\\\n",
       "\t 20120104 & 000005.SZ &  0.0047238201\\\\\n",
       "\t 20120104 & 000006.SZ &  0.0108104722\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | symbol &lt;chr&gt; | abnr_ret &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | 000001.SZ | -0.0187202985 |\n",
       "| 20120104 | 000002.SZ | -0.0008383009 |\n",
       "| 20120104 | 000004.SZ |  0.0114847517 |\n",
       "| 20120104 | 000005.SZ |  0.0047238201 |\n",
       "| 20120104 | 000006.SZ |  0.0108104722 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     symbol    abnr_ret     \n",
       "1 20120104 000001.SZ -0.0187202985\n",
       "2 20120104 000002.SZ -0.0008383009\n",
       "3 20120104 000004.SZ  0.0114847517\n",
       "4 20120104 000005.SZ  0.0047238201\n",
       "5 20120104 000006.SZ  0.0108104722"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, weight = capt/sum(capt)), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(weight*stkcd_ret), stkcd_ret, symbol), keyby = .(industry, date)\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ ind_ret))[1], beta = coef(lm(stkcd_ret ~ ind_ret))[2], ind_ret, stkcd_ret, symbol, date), keyby = .(industry, symbol)\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * ind_ret), keyby = .(date, symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 33. 每天每个股票对行业去除自身的超额收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与`Ex-31`类似，可完全参考`Ex-31`，此处不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>symbol</th><th scope=col>abnr_ret</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>000001.SZ</td><td>-0.019263254</td></tr>\n",
       "\t<tr><td>20120104</td><td>000002.SZ</td><td>-0.001687256</td></tr>\n",
       "\t<tr><td>20120104</td><td>000004.SZ</td><td> 0.011452958</td></tr>\n",
       "\t<tr><td>20120104</td><td>000005.SZ</td><td> 0.004647871</td></tr>\n",
       "\t<tr><td>20120104</td><td>000006.SZ</td><td> 0.010585465</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & symbol & abnr\\_ret\\\\\n",
       " <int> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 20120104 & 000001.SZ & -0.019263254\\\\\n",
       "\t 20120104 & 000002.SZ & -0.001687256\\\\\n",
       "\t 20120104 & 000004.SZ &  0.011452958\\\\\n",
       "\t 20120104 & 000005.SZ &  0.004647871\\\\\n",
       "\t 20120104 & 000006.SZ &  0.010585465\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | symbol &lt;chr&gt; | abnr_ret &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 20120104 | 000001.SZ | -0.019263254 |\n",
       "| 20120104 | 000002.SZ | -0.001687256 |\n",
       "| 20120104 | 000004.SZ |  0.011452958 |\n",
       "| 20120104 | 000005.SZ |  0.004647871 |\n",
       "| 20120104 | 000006.SZ |  0.010585465 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     symbol    abnr_ret    \n",
       "1 20120104 000001.SZ -0.019263254\n",
       "2 20120104 000002.SZ -0.001687256\n",
       "3 20120104 000004.SZ  0.011452958\n",
       "4 20120104 000005.SZ  0.004647871\n",
       "5 20120104 000006.SZ  0.010585465"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, ind_capt_outself =  sum(capt) - capt, capt), keyby = .(industry,date)\n",
    "    ][, .(ind_ret_outself = {\n",
    "         a <- vector()\n",
    "        for (i in 1:.N) {\n",
    "            a[i] <- sum((capt[-i]/ind_capt_outself[i]) * stkcd_ret[-i])\n",
    "        }\n",
    "        a\n",
    "    }, stkcd_ret, symbol),     \n",
    "      keyby = .(industry, date)\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ ind_ret_outself))[1], beta = coef(lm(stkcd_ret ~ ind_ret_outself))[2], ind_ret_outself, stkcd_ret, symbol, date), keyby = .(industry, symbol)\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * ind_ret_outself), keyby = .(date, symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 34. 每个股票每天对市场的超额收益率与对行业的超额收益率的相关系数如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### key 1\n",
    "此题为`Ex-30`和`Ex-32`的结合，最后合并了这两题的两个表，得出结果。可完全参考`Ex-30`和`Ex-32`，此处不做赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.946768976675315"
      ],
      "text/latex": [
       "0.946768976675315"
      ],
      "text/markdown": [
       "0.946768976675315"
      ],
      "text/plain": [
       "[1] 0.946769"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mkt <- data[, .(stkcd_ret = close/pre_close - 1, weight = capt/sum(capt), symbol), keyby = date\n",
    "    ][, .(mkt_ret = sum(weight/100*stkcd_ret), stkcd_ret, symbol), keyby = date\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ mkt_ret))[1], beta = coef(lm(stkcd_ret ~ mkt_ret))[2], mkt_ret, stkcd_ret, symbol, date)\n",
    "    ][, .(abnr_mkt_ret = stkcd_ret - alpha - beta * mkt_ret), keyby = .(date, symbol)]\n",
    "ind <- data[, .(stkcd_ret = close/pre_close - 1, symbol, weight = capt/sum(capt)), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(weight/100*stkcd_ret), stkcd_ret, symbol), keyby = .(industry, date)\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ ind_ret))[1], beta = coef(lm(stkcd_ret ~ ind_ret))[2], ind_ret, stkcd_ret, symbol, date), keyby = .(industry)\n",
    "    ][, .(abnr_ind_ret = stkcd_ret - alpha - beta * ind_ret), \n",
    "      keyby = .(date, symbol)]\n",
    "\n",
    "mkt[ind, on = .(date, symbol)\n",
    "   ][, cor(abnr_mkt_ret, abnr_ind_ret)]\n",
    "rm(mkt, ind)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题由于牵涉的变量生成较多，运用`.()`进行处理会比较容易出错，因为每行都要想清楚下一行需要哪些变量，需要提前提取一些变量。此处再提供一个较多利用`:=`进行处理的答案，还可以避免两个表进行合并的操作，整体代码比较整洁。基本思路与Key 1一致，可参考Key 1。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.946768976675301"
      ],
      "text/latex": [
       "0.946768976675301"
      ],
      "text/markdown": [
       "0.946768976675301"
      ],
      "text/plain": [
       "[1] 0.946769"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data_key2 <- data\n",
    "data_key2[, ':='(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt)), keyby = date\n",
    "    ][, mkt_ret := sum(mkt_weight/100*stkcd_ret), keyby = date\n",
    "    ][, ':='(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt)), keyby = .(industry,date)\n",
    "    ][, ind_ret := sum(ind_weight/100*stkcd_ret), keyby = .(industry, date)\n",
    "    ][, ':='(alpha_mkt = coef(lm(stkcd_ret ~ mkt_ret))[1], beta_mkt = coef(lm(stkcd_ret ~ mkt_ret))[2])\n",
    "    ][, ':='(alpha_ind = coef(lm(stkcd_ret ~ ind_ret))[1], beta_ind = coef(lm(stkcd_ret ~ ind_ret))[2]), keyby = .(industry)\n",
    "    ][, ':='(abnr_ind_ret = stkcd_ret - alpha_ind - beta_ind * ind_ret, abnr_mkt_ret = stkcd_ret - alpha_mkt - beta_mkt * mkt_ret), keyby = .(date, symbol)\n",
    "    ][, cor(abnr_mkt_ret, abnr_ind_ret)]\n",
    "rm(data_key2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 35. 每天有哪些行业的平均收益率超过市场平均收益率？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题主要计算每天的行业收益率`ind_ret`和市场收益率`mkt_ret`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 4</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>ind_ret</th><th scope=col>mkt_ret</th><th scope=col>industry</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120104</td><td>-0.008403388</td><td>-0.01581648</td><td>AERODEF </td></tr>\n",
       "\t<tr><td>20120104</td><td>-0.013057659</td><td>-0.01581648</td><td>AUTO    </td></tr>\n",
       "\t<tr><td>20120104</td><td>-0.006737250</td><td>-0.01581648</td><td>BANKS   </td></tr>\n",
       "\t<tr><td>20120104</td><td>-0.003713831</td><td>-0.01581648</td><td>ENERGY  </td></tr>\n",
       "\t<tr><td>20120104</td><td>-0.011340144</td><td>-0.01581648</td><td>HOUSEDUR</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 4\n",
       "\\begin{tabular}{llll}\n",
       " date & ind\\_ret & mkt\\_ret & industry\\\\\n",
       " <int> & <dbl> & <dbl> & <chr>\\\\\n",
       "\\hline\n",
       "\t 20120104 & -0.008403388 & -0.01581648 & AERODEF \\\\\n",
       "\t 20120104 & -0.013057659 & -0.01581648 & AUTO    \\\\\n",
       "\t 20120104 & -0.006737250 & -0.01581648 & BANKS   \\\\\n",
       "\t 20120104 & -0.003713831 & -0.01581648 & ENERGY  \\\\\n",
       "\t 20120104 & -0.011340144 & -0.01581648 & HOUSEDUR\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 4\n",
       "\n",
       "| date &lt;int&gt; | ind_ret &lt;dbl&gt; | mkt_ret &lt;dbl&gt; | industry &lt;chr&gt; |\n",
       "|---|---|---|---|\n",
       "| 20120104 | -0.008403388 | -0.01581648 | AERODEF  |\n",
       "| 20120104 | -0.013057659 | -0.01581648 | AUTO     |\n",
       "| 20120104 | -0.006737250 | -0.01581648 | BANKS    |\n",
       "| 20120104 | -0.003713831 | -0.01581648 | ENERGY   |\n",
       "| 20120104 | -0.011340144 | -0.01581648 | HOUSEDUR |\n",
       "\n"
      ],
      "text/plain": [
       "  date     ind_ret      mkt_ret     industry\n",
       "1 20120104 -0.008403388 -0.01581648 AERODEF \n",
       "2 20120104 -0.013057659 -0.01581648 AUTO    \n",
       "3 20120104 -0.006737250 -0.01581648 BANKS   \n",
       "4 20120104 -0.003713831 -0.01581648 ENERGY  \n",
       "5 20120104 -0.011340144 -0.01581648 HOUSEDUR"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, ind_weight = capt/sum(capt), capt), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(ind_weight*stkcd_ret), stkcd_ret, symbol, capt), keyby = .(industry, date)\n",
    "    ][, .(ind_ret, mkt_weight = capt/sum(capt), stkcd_ret, industry), keyby = date\n",
    "    ][, .(ind_ret, mkt_ret = sum(mkt_weight*stkcd_ret), industry), keyby = date\n",
    "    ][ind_ret > mkt_ret, unique(.SD)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票每天的收益率`stkcd_ret`和每个行业中各股票的流通市值权重`ind_weight`。\n",
    "- line 2 计算每个行业每天的收益率`ind_ret`。\n",
    "- line 3 计算每天市场中各股票的流通市值权重`mkt_weight`。\n",
    "- line 4 计算市场每天的收益率`mkt_ret`。\n",
    "- line 5 挑选出每天那些行业收益率`ind_ret`大于市场收益率`mkt_ret`的行业，并去重。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 36. 每天每个行业对市场的超额收益率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与`Ex-30`类似，此题最大的不同在于需要以行业`industry`和日期`date`，分组计算每个行业的收益率`ind_ret`和市场收益率`mkt_ret`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>abnr_ret</th><th scope=col>date</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td> 0.010403434</td><td>20120104</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.008206711</td><td>20120105</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.027298407</td><td>20120106</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.004859186</td><td>20120109</td></tr>\n",
       "\t<tr><td>AERODEF</td><td> 0.006746527</td><td>20120110</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " industry & abnr\\_ret & date\\\\\n",
       " <chr> & <dbl> & <int>\\\\\n",
       "\\hline\n",
       "\t AERODEF &  0.010403434 & 20120104\\\\\n",
       "\t AERODEF & -0.008206711 & 20120105\\\\\n",
       "\t AERODEF & -0.027298407 & 20120106\\\\\n",
       "\t AERODEF & -0.004859186 & 20120109\\\\\n",
       "\t AERODEF &  0.006746527 & 20120110\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| industry &lt;chr&gt; | abnr_ret &lt;dbl&gt; | date &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| AERODEF |  0.010403434 | 20120104 |\n",
       "| AERODEF | -0.008206711 | 20120105 |\n",
       "| AERODEF | -0.027298407 | 20120106 |\n",
       "| AERODEF | -0.004859186 | 20120109 |\n",
       "| AERODEF |  0.006746527 | 20120110 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry abnr_ret     date    \n",
       "1 AERODEF   0.010403434 20120104\n",
       "2 AERODEF  -0.008206711 20120105\n",
       "3 AERODEF  -0.027298407 20120106\n",
       "4 AERODEF  -0.004859186 20120109\n",
       "5 AERODEF   0.006746527 20120110"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, ind_weight = capt/sum(capt), capt), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(ind_weight*stkcd_ret), stkcd_ret, symbol, capt), keyby = .(industry, date)\n",
    "    ][, .(ind_ret, mkt_weight = capt/sum(capt), stkcd_ret, industry), keyby = date\n",
    "    ][, .(ind_ret, mkt_ret = sum(mkt_weight*stkcd_ret), industry), keyby = date\n",
    "    ][, unique(.SD)\n",
    "    ][, .(alpha = coef(lm(ind_ret ~ mkt_ret))[1], beta = coef(lm(ind_ret ~ mkt_ret))[2], ind_ret, mkt_ret, date), keyby = industry\n",
    "    ][, .(abnr_ret = ind_ret - alpha - beta*mkt_ret, date), by = industry\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 37. 每天每个行业对去除本行业后的市场超额收益是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与`Ex-31`类似，唯一不同点在于需要在计算完每个行业每天的收益率`ind_ret`之后，运用公式 $ ret^m_i = \\sum_{i = 1, i \\ne k}^{n-1} \\frac {capt_i \\cdot ret_i}{\\sum_{j = 1, j \\ne k}^{n-1} capt_j} $，计算去除本行业的市场收益率。可参考`Ex-31`，这里不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>abnr_ret</th><th scope=col>date</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td> 0.009792295</td><td>20120104</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.009881059</td><td>20120105</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.029005787</td><td>20120106</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>-0.007261510</td><td>20120109</td></tr>\n",
       "\t<tr><td>AERODEF</td><td> 0.005183683</td><td>20120110</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " industry & abnr\\_ret & date\\\\\n",
       " <chr> & <dbl> & <int>\\\\\n",
       "\\hline\n",
       "\t AERODEF &  0.009792295 & 20120104\\\\\n",
       "\t AERODEF & -0.009881059 & 20120105\\\\\n",
       "\t AERODEF & -0.029005787 & 20120106\\\\\n",
       "\t AERODEF & -0.007261510 & 20120109\\\\\n",
       "\t AERODEF &  0.005183683 & 20120110\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| industry &lt;chr&gt; | abnr_ret &lt;dbl&gt; | date &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| AERODEF |  0.009792295 | 20120104 |\n",
       "| AERODEF | -0.009881059 | 20120105 |\n",
       "| AERODEF | -0.029005787 | 20120106 |\n",
       "| AERODEF | -0.007261510 | 20120109 |\n",
       "| AERODEF |  0.005183683 | 20120110 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry abnr_ret     date    \n",
       "1 AERODEF   0.009792295 20120104\n",
       "2 AERODEF  -0.009881059 20120105\n",
       "3 AERODEF  -0.029005787 20120106\n",
       "4 AERODEF  -0.007261510 20120109\n",
       "5 AERODEF   0.005183683 20120110"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, ind_weight = capt/sum(capt), capt), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(ind_weight*stkcd_ret), stkcd_ret, symbol, capt), keyby = .(industry, date)\n",
    "    ][, .(ind_capt = sum(capt), ind_ret), keyby = .(industry, date)\n",
    "    ][, unique(.SD)\n",
    "    ][, .(ind_mkt_weight = ind_capt/(sum(ind_capt) - ind_capt), ind_ret, industry), keyby = date\n",
    "    ][, .(mkt_ret = sum(ind_mkt_weight * ind_ret) - ind_mkt_weight * ind_ret, ind_ret, industry), by = date\n",
    "    ][, .(alpha = coef(lm(ind_ret ~ mkt_ret))[1], beta = coef(lm(ind_ret ~ mkt_ret))[2], ind_ret, mkt_ret, industry, date)\n",
    "    ][, .(abnr_ret = ind_ret - alpha - beta*mkt_ret, date), by = industry\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 需要注意的是 line 4 中用了一个`unique(.SD)`的处理，因为原始数据是以每一只股票每一天作为最小观测的，但这里需要的是每一个行业每天的收益率和总体市值，在提取`ind_ret`和`ind_capt`时存在很多的重复观察，故而用了去重命令。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 38. 每天分别有多少股票是最近连续3个交易日上涨、下跌的？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题的关键点和难点在于，如何识别出连续三个交易日上涨和下跌。由于牵涉到行处理，所以最好的方法是在`data.table`语句中进行循环。本题运用了`logical`类向量在四则运算时`TRUE`为`1`，`FALSE`为`0`的特征，进行识别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>tag</th><th scope=col>stkcd_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120109</td><td>r3day_dn</td><td>535</td></tr>\n",
       "\t<tr><td>20120109</td><td>r3day_up</td><td> 16</td></tr>\n",
       "\t<tr><td>20120110</td><td>r3day_dn</td><td> 11</td></tr>\n",
       "\t<tr><td>20120110</td><td>r3day_up</td><td> 71</td></tr>\n",
       "\t<tr><td>20120111</td><td>r3day_dn</td><td>  2</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date & tag & stkcd\\_amount\\\\\n",
       " <int> & <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120109 & r3day\\_dn & 535\\\\\n",
       "\t 20120109 & r3day\\_up &  16\\\\\n",
       "\t 20120110 & r3day\\_dn &  11\\\\\n",
       "\t 20120110 & r3day\\_up &  71\\\\\n",
       "\t 20120111 & r3day\\_dn &   2\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| date &lt;int&gt; | tag &lt;chr&gt; | stkcd_amount &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| 20120109 | r3day_dn | 535 |\n",
       "| 20120109 | r3day_up |  16 |\n",
       "| 20120110 | r3day_dn |  11 |\n",
       "| 20120110 | r3day_up |  71 |\n",
       "| 20120111 | r3day_dn |   2 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     tag      stkcd_amount\n",
       "1 20120109 r3day_dn 535         \n",
       "2 20120109 r3day_up  16         \n",
       "3 20120110 r3day_dn  11         \n",
       "4 20120110 r3day_up  71         \n",
       "5 20120111 r3day_dn   2         "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1), keyby = .(symbol, date)\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        b1 <- stkcd_ret > 0\n",
    "        b2 <- stkcd_ret < 0\n",
    "        for (t in 1:.N) {\n",
    "            l[[t+3]] <- list(r3day_up = mean(b1[t:(t+2)]), r3day_dn = mean(b2[t:(t+2)]), date = date[t+3])\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }    \n",
    "    , keyby = .(symbol)\n",
    "    ][!is.na(date), .(stkcd_amount = uniqueN(symbol)), keyby = .(date, tag = ifelse(r3day_up == 1, \"r3day_up\", ifelse(r3day_dn == 1, \"r3day_dn\", \"others\")))\n",
    "    ][tag == \"r3day_dn\"|tag == \"r3day_up\"\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 首先计算出每一只股票每一天的收益率`stkcd_ret`。\n",
    "- line 2 是本题的关键.首先由于要对每一只股票进行对应处理，先用`keyby = .(symbol)`进行分组。接下来定义一个list`l`，接下来把每只股票每天的收益率`stkcd_ret`做一个判断，这一天的收益率`大于0`为`上涨`，收益率`小于0`为`下跌`，以此生成两列`logical`类型的变量`b1`和`b2`。在`b1`中观测如果为`TRUE`则表明该只股票这一天股价为`上涨`，反之如果为`FALSE`则为`下跌`；`b2`中的观测代表的意义与`b1`相反。接下来，从第一行到最后一行，设定一个循环的`t`值，由于是判断最近连续3个交易日是否涨跌，那么就从每只股票的第4个交易日`t+3`开始计算，因而有`l[[t+3]]`和`date[t+3]`；而后计算`b1`和`b2`最近三天的均值，分别为`r3day_up`和`r3day_dn`（为什么求均值会在后面 ）。由于每一次循环生成了三个变量的一次观测，所以将这一次观测生成一个`list`，而后对应到每一个`l`的每一天的观测中去，于是就有了 `l[[t+3]] <- list(r3day_up = mean(b1[t:(t+2)]), r3day_dn = mean(b2[t:(t+2)]), date = date[t+3])`。最后，需要对生成的`.N-3`行观测进行合并，在这里用到了`rbindlist(l)`。\n",
    "- line 3 则计算出了每一天当中最近三天上涨和下跌的股票数。首先以`!is.na(date)`去除`date`为`NA`的观测，因为当循环到`.N-2`时，`r3day_up`和`r3day_dn`还能生成观测，但`date`已无法生成观测，超出了循环的日期范围，故而会出现`NA`的情况；接下里在`by`中进行分组，需要生成一个`tag`变量，我们可以发现，`r3day_up`是最近三日是否为上涨判断的均值，如果最近三日皆为上涨，则`r3day_up`应该为`1`；同理可以推断`r3day_dn`，如果最近三日皆为下跌，则`r3day_dn`应为`1`。故而将`tag`设定为三种观测值`r3day_up`、`r3day_dn`以及`others`，用`ifelse`语句进行生成。而后根据`date`和`tag`分组计算，每天属于`r3day_up`、`r3day_dn`以及`others`的股票数量：`stkcd_amount = uniqueN(symbol)`。\n",
    "- line 4 最后挑选出`tag`为`r3day_up`和`r3day_dn`的行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 39. 每天分别有多少股票是最近连续3个交易日收益率超过当天市场平均收益率？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "思路与`Ex-38`基本一致，不同点在于需要先计算市场收益率`mkt_ret`和单只股票收益率`stkcd_ret`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>stkcd_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120109</td><td> 81</td></tr>\n",
       "\t<tr><td>20120110</td><td> 95</td></tr>\n",
       "\t<tr><td>20120111</td><td>785</td></tr>\n",
       "\t<tr><td>20120112</td><td>905</td></tr>\n",
       "\t<tr><td>20120113</td><td>392</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & stkcd\\_amount\\\\\n",
       " <int> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120109 &  81\\\\\n",
       "\t 20120110 &  95\\\\\n",
       "\t 20120111 & 785\\\\\n",
       "\t 20120112 & 905\\\\\n",
       "\t 20120113 & 392\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | stkcd_amount &lt;int&gt; |\n",
       "|---|---|\n",
       "| 20120109 |  81 |\n",
       "| 20120110 |  95 |\n",
       "| 20120111 | 785 |\n",
       "| 20120112 | 905 |\n",
       "| 20120113 | 392 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     stkcd_amount\n",
       "1 20120109  81         \n",
       "2 20120110  95         \n",
       "3 20120111 785         \n",
       "4 20120112 905         \n",
       "5 20120113 392         "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt), symbol), keyby = .(date)\n",
    "    ][, .(mkt_ret = sum(stkcd_ret * mkt_weight), symbol, stkcd_ret), keyby = .(date)\n",
    "    ][, .(rday_ret = ifelse(stkcd_ret > mkt_ret, 1, 0)), keyby = .(symbol, date)\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        for (t in 1:.N) {\n",
    "            l[[t+3]] <- list(r3day_ret = mean(rday_ret[t:(t+2)]), date = date[t+3])\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }, keyby = symbol\n",
    "    ][r3day_ret == 1 & !is.na(date), .(stkcd_amount = uniqueN(symbol)), keyby = date\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 和 line 2 分别计算出每日市场收益率`mkt_ret`和每日每只股票的收益率`stkcd_ret`。\n",
    "- line 3 则判断每日该股票是否超过该日市场收益率，超过为1，没超过为0，写入变量`rday_ret`中。\n",
    "- line 4 ~ line 5可完全参考`Ex-38`中的line 3 ~ line 4的解析，在此不作赘述。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 40. 每天分别有多少股票是最新5个交易日中至少有4个交易日的收益率超过当天市场平均收益率？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与`Ex-39`思路基本一致，可完全参考`Ex-39`，在此不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date</th><th scope=col>stkcd_amount</th></tr>\n",
       "\t<tr><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>20120111</td><td> 14</td></tr>\n",
       "\t<tr><td>20120112</td><td> 22</td></tr>\n",
       "\t<tr><td>20120113</td><td>184</td></tr>\n",
       "\t<tr><td>20120116</td><td> 48</td></tr>\n",
       "\t<tr><td>20120117</td><td> 18</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date & stkcd\\_amount\\\\\n",
       " <int> & <int>\\\\\n",
       "\\hline\n",
       "\t 20120111 &  14\\\\\n",
       "\t 20120112 &  22\\\\\n",
       "\t 20120113 & 184\\\\\n",
       "\t 20120116 &  48\\\\\n",
       "\t 20120117 &  18\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date &lt;int&gt; | stkcd_amount &lt;int&gt; |\n",
       "|---|---|\n",
       "| 20120111 |  14 |\n",
       "| 20120112 |  22 |\n",
       "| 20120113 | 184 |\n",
       "| 20120116 |  48 |\n",
       "| 20120117 |  18 |\n",
       "\n"
      ],
      "text/plain": [
       "  date     stkcd_amount\n",
       "1 20120111  14         \n",
       "2 20120112  22         \n",
       "3 20120113 184         \n",
       "4 20120116  48         \n",
       "5 20120117  18         "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt), symbol), keyby = .(date)\n",
    "    ][, .(mkt_ret = sum(stkcd_ret * mkt_weight), symbol, stkcd_ret), keyby = .(date)\n",
    "    ][, .(rday_ret = ifelse(stkcd_ret > mkt_ret, 1, 0)), keyby = .(symbol, date)\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        for (t in 1:.N) {\n",
    "            l[[t+5]] <- list(r3day_ret = mean(rday_ret[t:(t+4)]), date = date[t+5])\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }, keyby = symbol\n",
    "    ][r3day_ret > 0.8 & !is.na(date), .(stkcd_amount = uniqueN(symbol)), keyby = date\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 41. 每个月中，个股月收益超过市场月收益1倍以上的股票有哪些？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于分别计算个股月收益`stkcd_m_ret`和市场月收益`mkt_m_ret`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date_ym</th><th scope=col>symbol</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>201201</td><td>000009.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>000012.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>000017.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>000018.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>000030.SZ</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date\\_ym & symbol\\\\\n",
       " <chr> & <chr>\\\\\n",
       "\\hline\n",
       "\t 201201 & 000009.SZ\\\\\n",
       "\t 201201 & 000012.SZ\\\\\n",
       "\t 201201 & 000017.SZ\\\\\n",
       "\t 201201 & 000018.SZ\\\\\n",
       "\t 201201 & 000030.SZ\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date_ym &lt;chr&gt; | symbol &lt;chr&gt; |\n",
       "|---|---|\n",
       "| 201201 | 000009.SZ |\n",
       "| 201201 | 000012.SZ |\n",
       "| 201201 | 000017.SZ |\n",
       "| 201201 | 000018.SZ |\n",
       "| 201201 | 000030.SZ |\n",
       "\n"
      ],
      "text/plain": [
       "  date_ym symbol   \n",
       "1 201201  000009.SZ\n",
       "2 201201  000012.SZ\n",
       "3 201201  000017.SZ\n",
       "4 201201  000018.SZ\n",
       "5 201201  000030.SZ"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt), symbol), keyby = .(date)\n",
    "    ][, .(mkt_ret = sum(mkt_weight * stkcd_ret), stkcd_ret, symbol, date_ym = str_sub(date, start = 1, end = 6)), keyby = .(date)\n",
    "    ][, .(stkcd_m_ret = mean(stkcd_ret), date, mkt_ret), keyby = .(symbol, date_ym)\n",
    "    ][, .(stkcd_m_ret, symbol, date, mkt_m_ret = mean(mkt_ret)), keyby = .(date_ym)\n",
    "    ][stkcd_m_ret > 2*mkt_m_ret, .(symbol = unique(symbol)), keyby = .(date_ym)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每日个股收益率`stkcd_ret`和个股每日在市场中的市值权重`mkt_weight`。\n",
    "- line 2 计算出每日的市场收益率`mkt_ret`，并生成一个精确到某年某月的时间变量`date_ym`，这里用了`str_sub`对`date`前6位进行提取。\n",
    "- line 3 以`symbol`和`date_ym`分组计算个股月收益率`stkcd_m_ret`。\n",
    "- line 4 以`date_ym`分组计算市场平均月收益率`mkt_m_ret`。\n",
    "- line 5 判断语句挑选出每月收益大于市场收益1倍的股票：`stkcd_m_ret > 2*mkt_m_ret`。`keyby = .(date_ym)`标记出年月，而后用`unique(symbol)`对股票代码去重。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 42. 每个月中，个股月收益超过行业月收益1倍以上的股票有哪些？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题与`Ex-41`类似，可完全参考，此处不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date_ym</th><th scope=col>symbol</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>201201</td><td>000768.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>002297.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>300034.SZ</td></tr>\n",
       "\t<tr><td>201201</td><td>600038.SH</td></tr>\n",
       "\t<tr><td>201201</td><td>600391.SH</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " date\\_ym & symbol\\\\\n",
       " <chr> & <chr>\\\\\n",
       "\\hline\n",
       "\t 201201 & 000768.SZ\\\\\n",
       "\t 201201 & 002297.SZ\\\\\n",
       "\t 201201 & 300034.SZ\\\\\n",
       "\t 201201 & 600038.SH\\\\\n",
       "\t 201201 & 600391.SH\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| date_ym &lt;chr&gt; | symbol &lt;chr&gt; |\n",
       "|---|---|\n",
       "| 201201 | 000768.SZ |\n",
       "| 201201 | 002297.SZ |\n",
       "| 201201 | 300034.SZ |\n",
       "| 201201 | 600038.SH |\n",
       "| 201201 | 600391.SH |\n",
       "\n"
      ],
      "text/plain": [
       "  date_ym symbol   \n",
       "1 201201  000768.SZ\n",
       "2 201201  002297.SZ\n",
       "3 201201  300034.SZ\n",
       "4 201201  600038.SH\n",
       "5 201201  600391.SH"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt), symbol), keyby = .(industry, date)\n",
    "    ][, .(ind_ret = sum(ind_weight * stkcd_ret), stkcd_ret, symbol, date_ym = str_sub(date, start = 1, end = 6)), keyby = .(industry, date)\n",
    "    ][, .(stkcd_m_ret = mean(stkcd_ret), date, ind_ret, industry), keyby = .(symbol, date_ym)\n",
    "    ][, .(stkcd_m_ret, symbol, date, ind_m_ret = mean(ind_ret)), keyby = .(date_ym, industry)\n",
    "    ][stkcd_m_ret > 2*ind_m_ret, .(symbol = unique(symbol)), keyby = .(date_ym)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 43. 每个股票的收益率对市场收益率的相关系数最高的10个股票是哪些？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于分别求出每只股票和市场每天的收益率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n",
      "Warning message in cor(stkcd_ret, mkt_ret):\n",
      "“the standard deviation is zero”\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>cor_coef</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>300331.SZ</td><td>1.0000000</td></tr>\n",
       "\t<tr><td>600508.SH</td><td>0.9279554</td></tr>\n",
       "\t<tr><td>601101.SH</td><td>0.9097758</td></tr>\n",
       "\t<tr><td>000685.SZ</td><td>0.8979148</td></tr>\n",
       "\t<tr><td>601666.SH</td><td>0.8939529</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & cor\\_coef\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 300331.SZ & 1.0000000\\\\\n",
       "\t 600508.SH & 0.9279554\\\\\n",
       "\t 601101.SH & 0.9097758\\\\\n",
       "\t 000685.SZ & 0.8979148\\\\\n",
       "\t 601666.SH & 0.8939529\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | cor_coef &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| 300331.SZ | 1.0000000 |\n",
       "| 600508.SH | 0.9279554 |\n",
       "| 601101.SH | 0.9097758 |\n",
       "| 000685.SZ | 0.8979148 |\n",
       "| 601666.SH | 0.8939529 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    cor_coef \n",
       "1 300331.SZ 1.0000000\n",
       "2 600508.SH 0.9279554\n",
       "3 601101.SH 0.9097758\n",
       "4 000685.SZ 0.8979148\n",
       "5 601666.SH 0.8939529"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt), symbol), keyby = .(date)\n",
    "    ][, .(stkcd_ret, mkt_ret = sum(mkt_weight * stkcd_ret), symbol), keyby = .(date)\n",
    "    ][, .(cor_coef = cor(stkcd_ret, mkt_ret)), keyby = .(symbol)\n",
    "    ][order(-cor_coef), .SD[1:10]\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票每天的收益率`stkcd_ret`和每天的市值加权比例`mkt_weight`。\n",
    "- line 2 计算每天的市场收益率`mkt_ret`。\n",
    "- line 3 计算股票收益率与市场收益率的相关系数`cor_coef`，这里需要以`symbol`进行分组。\n",
    "- line 4 按照`cor_coef`进行降序排列，最后挑选出前十行。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 44. 每个行业日收益率的历史波动率是多少？（用日收益率计算标准差）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于计算行业日收益率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>ind_vol</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>0.017499519</td></tr>\n",
       "\t<tr><td>AIRLINE</td><td>0.017127041</td></tr>\n",
       "\t<tr><td>AUTO   </td><td>0.015096513</td></tr>\n",
       "\t<tr><td>BANKS  </td><td>0.008105859</td></tr>\n",
       "\t<tr><td>BEV    </td><td>0.016407569</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " industry & ind\\_vol\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 0.017499519\\\\\n",
       "\t AIRLINE & 0.017127041\\\\\n",
       "\t AUTO    & 0.015096513\\\\\n",
       "\t BANKS   & 0.008105859\\\\\n",
       "\t BEV     & 0.016407569\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| industry &lt;chr&gt; | ind_vol &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| AERODEF | 0.017499519 |\n",
       "| AIRLINE | 0.017127041 |\n",
       "| AUTO    | 0.015096513 |\n",
       "| BANKS   | 0.008105859 |\n",
       "| BEV     | 0.016407569 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry ind_vol    \n",
       "1 AERODEF  0.017499519\n",
       "2 AIRLINE  0.017127041\n",
       "3 AUTO     0.015096513\n",
       "4 BANKS    0.008105859\n",
       "5 BEV      0.016407569"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt), symbol), keyby = .(industry, date)\n",
    "    ][, .(ind_ret = sum(stkcd_ret * ind_weight)), keyby = .(industry, date)\n",
    "    ][, .(ind_vol = sd(ind_ret)), keyby = .(industry)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算个股每日收益率`stkcd_ret`和行业市值加权`ind_weight`。\n",
    "- line 2 计算每个行业每日收益率`ind_ret`。\n",
    "- line 3 计算行业波动率`ind_vol`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 45. 各个行业的日收益率的相关系数矩阵如何？哪两个行业相关性最高、最低？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于计算相关系数矩阵时的数据结构的变换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'HDWRSEMI'</li><li>'ELECEQP'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'HDWRSEMI'\n",
       "\\item 'ELECEQP'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'HDWRSEMI'\n",
       "2. 'ELECEQP'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] \"HDWRSEMI\" \"ELECEQP\" "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       ".list-inline {list-style: none; margin:0; padding: 0}\n",
       ".list-inline>li {display: inline-block}\n",
       ".list-inline>li:not(:last-child)::after {content: \"\\00b7\"; padding: 0 .5ex}\n",
       "</style>\n",
       "<ol class=list-inline><li>'BEV'</li><li>'BANKS'</li></ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'BEV'\n",
       "\\item 'BANKS'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'BEV'\n",
       "2. 'BANKS'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] \"BEV\"   \"BANKS\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cor.coef <- data[, .(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt), symbol, capt), keyby = .(industry, date)\n",
    "    ][, .(ind_ret = sum(stkcd_ret * ind_weight)), keyby = .(industry, date)\n",
    "    ][, dcast(.SD, date ~ industry, value.var = \"ind_ret\")\n",
    "    ][, cor(.SD[, -1])]\n",
    "\n",
    "which(max(cor.coef[cor.coef != 1]) == cor.coef, arr.ind = TRUE) %>% rownames()\n",
    "\n",
    "which(min(cor.coef) == cor.coef, arr.ind = TRUE) %>% rownames()\n",
    "\n",
    "rm(cor.coef)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 和 line 2 首先按照常规计算股票每日收益率`stkcd_ret`和行业内各股流通市值加权`ind_weight`，进而计算行业每日的收益率`ind_ret`。\n",
    "- line 3 为本题的关键，这里对数据集进行reshape，用`dcast`从一个长的数据集变成一个宽的数据集，保留`date`这样一个变量，将`industry`变量里面的观测全部变成变量，即`date ~ industry`。而后在生成的这一系列变量里面填充各行业的收益率`ind_ret`，即`value.var = \"ind_ret\"`。\n",
    "- line 4 计算每个行业收益率之间的相关系数矩阵：`cor(.SD[, -1])`\n",
    "- 接下来的两行就是挑选出相关性最高和最低的两个行业，因为生成的数据格式是矩阵（相关系数矩阵），我们要将其中最大的相关系数挑选出来，并且需要知道这两个行业的定位和名称，首先用判断语句`max(cor.coef[cor.coef != 1]) == cor.coef`挑选出最大的那个相关系数（注：需要删除相关系数为1的那些部分，因为这些是自身与自身的相关系数），而由于需要定位这两个行业，那么输出格式还需要是一个保留行名和列名的矩阵，那么需要在`which`这个函数中加入`arr.ind = TRUE`这个argument。最后用`rownames`返回两个行业的名称。同理可应用在相关系数最低的两个行业的选取。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 46. 各个行业的收益率对市场收益率的相关系数由高到低排列如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题与`Ex-43`类似，可完全参考，此处不作赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>cor_coef</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>TRDDIST</td><td>0.9519013</td></tr>\n",
       "\t<tr><td>MACH   </td><td>0.9511527</td></tr>\n",
       "\t<tr><td>CHEM   </td><td>0.9455305</td></tr>\n",
       "\t<tr><td>ELECEQP</td><td>0.9451663</td></tr>\n",
       "\t<tr><td>CNSTENG</td><td>0.9446049</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " industry & cor\\_coef\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t TRDDIST & 0.9519013\\\\\n",
       "\t MACH    & 0.9511527\\\\\n",
       "\t CHEM    & 0.9455305\\\\\n",
       "\t ELECEQP & 0.9451663\\\\\n",
       "\t CNSTENG & 0.9446049\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| industry &lt;chr&gt; | cor_coef &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| TRDDIST | 0.9519013 |\n",
       "| MACH    | 0.9511527 |\n",
       "| CHEM    | 0.9455305 |\n",
       "| ELECEQP | 0.9451663 |\n",
       "| CNSTENG | 0.9446049 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry cor_coef \n",
       "1 TRDDIST  0.9519013\n",
       "2 MACH     0.9511527\n",
       "3 CHEM     0.9455305\n",
       "4 ELECEQP  0.9451663\n",
       "5 CNSTENG  0.9446049"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, ind_weight = capt/sum(capt), capt), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(ind_weight*stkcd_ret), stkcd_ret, symbol, capt), keyby = .(industry, date)\n",
    "    ][, .(ind_ret, mkt_weight = capt/sum(capt), stkcd_ret, industry), keyby = date\n",
    "    ][, .(ind_ret, mkt_ret = sum(mkt_weight*stkcd_ret), industry), keyby = date\n",
    "    ][, unique(.SD)\n",
    "    ][, .(cor_coef = cor(ind_ret, mkt_ret)), keyby = .(industry)\n",
    "    ][order(-cor_coef)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 47. 每个月总成交额比上个月下降幅度最大的行业是哪个？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题的含义笔者理解为：在每个月选择出一个行业，这个行业和其他股票相比这个月比上个月成交额下降最大。关键操作在于计算每个行业每个月对上个月成交额的变化`dn_m_range`，以及挑选下降幅度最大的那一个行业。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 4 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>date_ym</th><th scope=col>industry</th><th scope=col>dn_m_range</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>201203</td><td>CONMAT </td><td> -21752549725</td></tr>\n",
       "\t<tr><td>201204</td><td>MTLMIN </td><td>-165775634730</td></tr>\n",
       "\t<tr><td>201205</td><td>REALEST</td><td>  -4528375663</td></tr>\n",
       "\t<tr><td>201206</td><td>MTLMIN </td><td>-123271994315</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 4 × 3\n",
       "\\begin{tabular}{lll}\n",
       " date\\_ym & industry & dn\\_m\\_range\\\\\n",
       " <chr> & <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 201203 & CONMAT  &  -21752549725\\\\\n",
       "\t 201204 & MTLMIN  & -165775634730\\\\\n",
       "\t 201205 & REALEST &   -4528375663\\\\\n",
       "\t 201206 & MTLMIN  & -123271994315\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 4 × 3\n",
       "\n",
       "| date_ym &lt;chr&gt; | industry &lt;chr&gt; | dn_m_range &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| 201203 | CONMAT  |  -21752549725 |\n",
       "| 201204 | MTLMIN  | -165775634730 |\n",
       "| 201205 | REALEST |   -4528375663 |\n",
       "| 201206 | MTLMIN  | -123271994315 |\n",
       "\n"
      ],
      "text/plain": [
       "  date_ym industry dn_m_range   \n",
       "1 201203  CONMAT    -21752549725\n",
       "2 201204  MTLMIN   -165775634730\n",
       "3 201205  REALEST    -4528375663\n",
       "4 201206  MTLMIN   -123271994315"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(date_ym = str_sub(date, start = 1, end = 6), amount), keyby = .(industry, date)\n",
    "    ][, .(ind_m_amount = sum(amount)), keyby = .(industry, date_ym)\n",
    "    ][, .(dn_m_range = {\n",
    "        a <- vector()\n",
    "        for (t in 2:.N) {\n",
    "            a[t] <- ind_m_amount[t] - ind_m_amount[t-1]\n",
    "        }\n",
    "        a \n",
    "    }, date_ym = date_ym), keyby = industry\n",
    "    ][!is.na(dn_m_range), .SD[min(dn_m_range) == dn_m_range & dn_m_range < 0], keyby = date_ym]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 生成一个变量表示年份和月份`date_ym`。\n",
    "- line 2 计算每个行业每个月的总成交额`ind_m_amount`。\n",
    "- line 3 计算每个行业这个月相比于上个月成交额变动的幅度`dn_m_range`，由于牵涉到行之间的操作，这里选择根据行业分组计算，对本月的成交额减去上月的总成交额`ind_m_amount[t] - ind_m_amount[t-1]`进行循环计算。\n",
    "- line 4 最后我们需要选择出每个月中相比于上个月总成交额下降幅度最大的那个行业，根据“先i，再by，最后j”的原则，首先我们去除`dn_m_range`为NA的观测，而后根据年月`date_ym`这个变量进行分组，最后还需要挑选出下降幅度最大的那个行业，就需要在.SD中再进行一次筛选，即`.SD[min(dn_m_range) == dn_m_range & dn_m_range < 0]`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 48. 数据当中各个股票的最大回撤幅度是多少？（最大回撤是从一个高点到低点的降幅的最大值）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于如何处置最高点减去最低点。由于最大回撤幅度定义在于从一个高点到之后的低点的最大值，那么需要将每一天的最高点和之后所有的最低点进行比较，而后选择最大降幅。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>mkt_rtrt</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>000001.SZ</td><td>3.33</td></tr>\n",
       "\t<tr><td>000002.SZ</td><td>0.92</td></tr>\n",
       "\t<tr><td>000004.SZ</td><td>2.12</td></tr>\n",
       "\t<tr><td>000005.SZ</td><td>1.74</td></tr>\n",
       "\t<tr><td>000006.SZ</td><td>1.47</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & mkt\\_rtrt\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 000001.SZ & 3.33\\\\\n",
       "\t 000002.SZ & 0.92\\\\\n",
       "\t 000004.SZ & 2.12\\\\\n",
       "\t 000005.SZ & 1.74\\\\\n",
       "\t 000006.SZ & 1.47\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | mkt_rtrt &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| 000001.SZ | 3.33 |\n",
       "| 000002.SZ | 0.92 |\n",
       "| 000004.SZ | 2.12 |\n",
       "| 000005.SZ | 1.74 |\n",
       "| 000006.SZ | 1.47 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    mkt_rtrt\n",
       "1 000001.SZ 3.33    \n",
       "2 000002.SZ 0.92    \n",
       "3 000004.SZ 2.12    \n",
       "4 000005.SZ 1.74    \n",
       "5 000006.SZ 1.47    "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(mkt_rtrt_d = {\n",
    "    a <- vector()\n",
    "    for (i in 1:.N) {\n",
    "        a[i] <- max(high[i] - low[i:.N])\n",
    "    }\n",
    "    a\n",
    "    }), by = .(symbol)\n",
    "    ][, .(mkt_rtrt = max(mkt_rtrt_d)), by = .(symbol)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 首先用每一天的最高点`high[i]`，遍历之后所有的的`low[i:.N]`，得到一个每一天的最大降幅`mkt_rtrt_d = max(high[i]-low[i:.N])`。\n",
    "- line 2 再在每一只股票每一天的最大降幅中选择一个最大降幅`mkt_rtrt`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 49. 每只股票的胜率是多少？（胜率是每天收益率为正数的概率）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于以股票进行分组计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>gain_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>000001.SZ</td><td>0.4188034</td></tr>\n",
       "\t<tr><td>000002.SZ</td><td>0.4700855</td></tr>\n",
       "\t<tr><td>000004.SZ</td><td>0.5128205</td></tr>\n",
       "\t<tr><td>000005.SZ</td><td>0.1538462</td></tr>\n",
       "\t<tr><td>000006.SZ</td><td>0.5128205</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & gain\\_ratio\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 000001.SZ & 0.4188034\\\\\n",
       "\t 000002.SZ & 0.4700855\\\\\n",
       "\t 000004.SZ & 0.5128205\\\\\n",
       "\t 000005.SZ & 0.1538462\\\\\n",
       "\t 000006.SZ & 0.5128205\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | gain_ratio &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| 000001.SZ | 0.4188034 |\n",
       "| 000002.SZ | 0.4700855 |\n",
       "| 000004.SZ | 0.5128205 |\n",
       "| 000005.SZ | 0.1538462 |\n",
       "| 000006.SZ | 0.5128205 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    gain_ratio\n",
       "1 000001.SZ 0.4188034 \n",
       "2 000002.SZ 0.4700855 \n",
       "3 000004.SZ 0.5128205 \n",
       "4 000005.SZ 0.1538462 \n",
       "5 000006.SZ 0.5128205 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1), keyby = .(symbol, date)\n",
    "    ][, .(gain_ratio = sum(stkcd_ret > 0)/.N), keyby = symbol\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票每天的收益率`stkcd_ret`\n",
    "- line 2 以股票代码`symbol`分组计算胜率，`gain_ratio = sum(stkcd_ret > 0)/.N`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 50. 每只股票的盈亏比是多少？（盈亏比是正收益之和与负收益之和的比值的绝对值）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于首先标记出每天每只股票是正收益还是负收益，而后对正负收益分别进行加总，而后将这两个正负收益分别取绝对值进行对比。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>gl_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>000001.SZ</td><td>0.9718614</td></tr>\n",
       "\t<tr><td>000002.SZ</td><td>1.2796856</td></tr>\n",
       "\t<tr><td>000004.SZ</td><td>1.0828690</td></tr>\n",
       "\t<tr><td>000005.SZ</td><td>0.5532427</td></tr>\n",
       "\t<tr><td>000006.SZ</td><td>1.3719934</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & gl\\_ratio\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t 000001.SZ & 0.9718614\\\\\n",
       "\t 000002.SZ & 1.2796856\\\\\n",
       "\t 000004.SZ & 1.0828690\\\\\n",
       "\t 000005.SZ & 0.5532427\\\\\n",
       "\t 000006.SZ & 1.3719934\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | gl_ratio &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| 000001.SZ | 0.9718614 |\n",
       "| 000002.SZ | 1.2796856 |\n",
       "| 000004.SZ | 1.0828690 |\n",
       "| 000005.SZ | 0.5532427 |\n",
       "| 000006.SZ | 1.3719934 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    gl_ratio \n",
       "1 000001.SZ 0.9718614\n",
       "2 000002.SZ 1.2796856\n",
       "3 000004.SZ 1.0828690\n",
       "4 000005.SZ 0.5532427\n",
       "5 000006.SZ 1.3719934"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1), keyby = .(symbol, date)\n",
    "    ][, .(stkcd_ret_pn = sum(stkcd_ret)), keyby = .(symbol, tag = ifelse(stkcd_ret > 0, \"gain\", \"loss\"))\n",
    "    ][, .(gl_ratio = abs(stkcd_ret_pn[1]/stkcd_ret_pn[.N])), keyby = symbol\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票每天的收益率`stkcd_ret`。\n",
    "- line 2 首先计算出一个分组变量`tag`，里面有`gain`和`loss`两个观测值，如果`stkcd_ret > 0`，则为`gain`，反之则为`loss`；而后分别对`gain`和`loss`的部分进行加总，生成累计收益`stkcd_ret_pn`。\n",
    "- line 3 计算二者绝对值的比例`gl_ratio`，这里每只股票都有两个`stkcd_ret_pn`的观测，所以用`abs(stkcd_ret_pn[1]/stkcd_ret_pn[.N])`计算出盈亏比。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 51. 市场的胜率是多少？（市场收益率为正的概率）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于分别计算出市场正负收益率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 1 × 1</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>gain_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>0.4871795</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 1 × 1\n",
       "\\begin{tabular}{l}\n",
       " gain\\_ratio\\\\\n",
       " <dbl>\\\\\n",
       "\\hline\n",
       "\t 0.4871795\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 1 × 1\n",
       "\n",
       "| gain_ratio &lt;dbl&gt; |\n",
       "|---|\n",
       "| 0.4871795 |\n",
       "\n"
      ],
      "text/plain": [
       "  gain_ratio\n",
       "1 0.4871795 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt)), keyby = date\n",
    "    ][, .(mkt_ret = sum(stkcd_ret * mkt_weight)), keyby = date\n",
    "    ][, .(gain_ratio = sum(mkt_ret > 0)/.N)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算个股每日收益率`stkcd_ret`，以及个股每日流通市值在市场中的权重`mkt_weight`。\n",
    "- line 2 计算市场每日收益率`mkt_ret`。\n",
    "- line 3 计算市场的胜率`gain_ratio`，这里直接在`sum`中判断出市场收益率大于0的那些观测与所有观测进行求商，`sum(mkt_ret > 0/.N)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 52. 市场的盈亏比是多少？（市场中每个股票的市值加权正收益和市值加权负收益之比）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于分别计算市场市值加权正收益与负收益。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 1 × 1</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>gl_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>1.147373</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 1 × 1\n",
       "\\begin{tabular}{l}\n",
       " gl\\_ratio\\\\\n",
       " <dbl>\\\\\n",
       "\\hline\n",
       "\t 1.147373\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 1 × 1\n",
       "\n",
       "| gl_ratio &lt;dbl&gt; |\n",
       "|---|\n",
       "| 1.147373 |\n",
       "\n"
      ],
      "text/plain": [
       "  gl_ratio\n",
       "1 1.147373"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, mkt_weight = capt/sum(capt)), keyby = date\n",
    "    ][, .(mkt_ret = sum(stkcd_ret * mkt_weight)), keyby = date\n",
    "    ][, .(mkt_ret_pn = sum(mkt_ret)), keyby = .(tag = ifelse(mkt_ret > 0, \"gain\", \"loss\"))\n",
    "    ][, .(gl_ratio = abs(mkt_ret_pn[1]/mkt_ret_pn[.N]))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票的日收益率`stkcd_ret`，以及个股每日流通市值在市场中的权重`mkt_weight`。\n",
    "- line 2 计算市场日收益率`mkt_ret`。\n",
    "- line 3 先计算出一个分组变量`tag`，这个变量标记出市场日收益率是`gain`还是`loss`，而后分别计算窗口期正收益之和和负收益之和`mkt_ret_pn`。\n",
    "- line 4 计算盈亏比`gl_ratio`：`abs(mkt_ret_pn[1]/mkt_ret_pn[.N])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 53. 每个行业的胜率是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题与`Ex-51`类似，可完全参考，此处不做赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>gain_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>0.5213675</td></tr>\n",
       "\t<tr><td>AIRLINE</td><td>0.4786325</td></tr>\n",
       "\t<tr><td>AUTO   </td><td>0.4700855</td></tr>\n",
       "\t<tr><td>BANKS  </td><td>0.4615385</td></tr>\n",
       "\t<tr><td>BEV    </td><td>0.5470085</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " industry & gain\\_ratio\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 0.5213675\\\\\n",
       "\t AIRLINE & 0.4786325\\\\\n",
       "\t AUTO    & 0.4700855\\\\\n",
       "\t BANKS   & 0.4615385\\\\\n",
       "\t BEV     & 0.5470085\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| industry &lt;chr&gt; | gain_ratio &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| AERODEF | 0.5213675 |\n",
       "| AIRLINE | 0.4786325 |\n",
       "| AUTO    | 0.4700855 |\n",
       "| BANKS   | 0.4615385 |\n",
       "| BEV     | 0.5470085 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry gain_ratio\n",
       "1 AERODEF  0.5213675 \n",
       "2 AIRLINE  0.4786325 \n",
       "3 AUTO     0.4700855 \n",
       "4 BANKS    0.4615385 \n",
       "5 BEV      0.5470085 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt)), keyby = .(industry, date)\n",
    "    ][, .(ind_ret = sum(stkcd_ret * ind_weight)), keyby = .(industry, date)\n",
    "    ][, .(gain_ratio = sum(ind_ret > 0)/.N), keyby = industry\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 54. 每个行业的盈亏比是多少？（行业盈亏比是行业内每个股票的市值加权的正收益率和市值加权的负收益率之比）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题与`Ex-52`类似，可完全参考，此处不做赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>gl_ratio</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>0.9973617</td></tr>\n",
       "\t<tr><td>AIRLINE</td><td>1.0715381</td></tr>\n",
       "\t<tr><td>AUTO   </td><td>1.1206090</td></tr>\n",
       "\t<tr><td>BANKS  </td><td>1.0158729</td></tr>\n",
       "\t<tr><td>BEV    </td><td>1.2812672</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " industry & gl\\_ratio\\\\\n",
       " <chr> & <dbl>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 0.9973617\\\\\n",
       "\t AIRLINE & 1.0715381\\\\\n",
       "\t AUTO    & 1.1206090\\\\\n",
       "\t BANKS   & 1.0158729\\\\\n",
       "\t BEV     & 1.2812672\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| industry &lt;chr&gt; | gl_ratio &lt;dbl&gt; |\n",
       "|---|---|\n",
       "| AERODEF | 0.9973617 |\n",
       "| AIRLINE | 1.0715381 |\n",
       "| AUTO    | 1.1206090 |\n",
       "| BANKS   | 1.0158729 |\n",
       "| BEV     | 1.2812672 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry gl_ratio \n",
       "1 AERODEF  0.9973617\n",
       "2 AIRLINE  1.0715381\n",
       "3 AUTO     1.1206090\n",
       "4 BANKS    1.0158729\n",
       "5 BEV      1.2812672"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, ind_weight = capt/sum(capt)), keyby = .(industry, date)\n",
    "    ][, .(ind_ret = sum(stkcd_ret * ind_weight)), keyby = .(industry, date)\n",
    "    ][, .(ind_ret_pn = sum(ind_ret)), keyby = .(industry, tag = ifelse(ind_ret > 0, \"gain\", \"loss\"))\n",
    "    ][, .(gl_ratio = abs(ind_ret_pn[1]/ind_ret_pn[.N])), keyby = industry\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 55. 是否存在股票的月成交额超过所在行业当月中某天一天总成交额的情况？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于计算出每只股票每月的成交额，而后计算每个行业每天成交额，二者进行对比。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 6</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>date</th><th scope=col>stkcd_m_amount</th><th scope=col>ind_d_amount</th><th scope=col>date_ym</th><th scope=col>symbol</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;chr&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td>1460111191</td><td>493331236</td><td>201201</td><td>000768.SZ</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td> 976641005</td><td>493331236</td><td>201201</td><td>002025.SZ</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td>1125117546</td><td>493331236</td><td>201201</td><td>600118.SH</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td>1522650678</td><td>493331236</td><td>201201</td><td>600316.SH</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td> 675067956</td><td>493331236</td><td>201201</td><td>600879.SH</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 6\n",
       "\\begin{tabular}{llllll}\n",
       " industry & date & stkcd\\_m\\_amount & ind\\_d\\_amount & date\\_ym & symbol\\\\\n",
       " <chr> & <int> & <dbl> & <dbl> & <chr> & <chr>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 20120104 & 1460111191 & 493331236 & 201201 & 000768.SZ\\\\\n",
       "\t AERODEF & 20120104 &  976641005 & 493331236 & 201201 & 002025.SZ\\\\\n",
       "\t AERODEF & 20120104 & 1125117546 & 493331236 & 201201 & 600118.SH\\\\\n",
       "\t AERODEF & 20120104 & 1522650678 & 493331236 & 201201 & 600316.SH\\\\\n",
       "\t AERODEF & 20120104 &  675067956 & 493331236 & 201201 & 600879.SH\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 6\n",
       "\n",
       "| industry &lt;chr&gt; | date &lt;int&gt; | stkcd_m_amount &lt;dbl&gt; | ind_d_amount &lt;dbl&gt; | date_ym &lt;chr&gt; | symbol &lt;chr&gt; |\n",
       "|---|---|---|---|---|---|\n",
       "| AERODEF | 20120104 | 1460111191 | 493331236 | 201201 | 000768.SZ |\n",
       "| AERODEF | 20120104 |  976641005 | 493331236 | 201201 | 002025.SZ |\n",
       "| AERODEF | 20120104 | 1125117546 | 493331236 | 201201 | 600118.SH |\n",
       "| AERODEF | 20120104 | 1522650678 | 493331236 | 201201 | 600316.SH |\n",
       "| AERODEF | 20120104 |  675067956 | 493331236 | 201201 | 600879.SH |\n",
       "\n"
      ],
      "text/plain": [
       "  industry date     stkcd_m_amount ind_d_amount date_ym symbol   \n",
       "1 AERODEF  20120104 1460111191     493331236    201201  000768.SZ\n",
       "2 AERODEF  20120104  976641005     493331236    201201  002025.SZ\n",
       "3 AERODEF  20120104 1125117546     493331236    201201  600118.SH\n",
       "4 AERODEF  20120104 1522650678     493331236    201201  600316.SH\n",
       "5 AERODEF  20120104  675067956     493331236    201201  600879.SH"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(date_ym = str_sub(date, start = 1, end = 6), amount, industry), keyby = .(symbol, date)\n",
    "    ][, .(stkcd_m_amount = sum(amount), amount, industry, date), keyby = .(date_ym, symbol)\n",
    "    ][, .(stkcd_m_amount, ind_d_amount = sum(amount), date_ym, symbol), keyby = .(industry, date)\n",
    "    ][stkcd_m_amount > ind_d_amount\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 生成一个仅表示年和月的变量`date_ym`，即截取`date`变量前6位字符。\n",
    "- line 2 根据所生成的变量`date_ym`和个股`symbol`，分组计算每只股票每个月的成交额`stkcd_m_amount`。\n",
    "- line 3 再根据行业`industry`和日期`date`，分组计算每个行业的日成交额`ind_d_amount`。\n",
    "- line 4 将这两个变量进行对比找出股票月成交额大于行业日成交额的部分：`stkcd_m_amount > ind_d_amount`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 56. 每天每个行业编入、编出的股票各有多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题的关键在于如何确定每天每个行业编入编出的股票有多少只，可以将其转化为交集问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>export</th><th scope=col>import</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>0</td><td>0</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>0</td><td>0</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>0</td><td>0</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>0</td><td>0</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>0</td><td>0</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " industry & export & import\\\\\n",
       " <chr> & <int> & <int>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 0 & 0\\\\\n",
       "\t AERODEF & 0 & 0\\\\\n",
       "\t AERODEF & 0 & 0\\\\\n",
       "\t AERODEF & 0 & 0\\\\\n",
       "\t AERODEF & 0 & 0\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| industry &lt;chr&gt; | export &lt;int&gt; | import &lt;int&gt; |\n",
       "|---|---|---|\n",
       "| AERODEF | 0 | 0 |\n",
       "| AERODEF | 0 | 0 |\n",
       "| AERODEF | 0 | 0 |\n",
       "| AERODEF | 0 | 0 |\n",
       "| AERODEF | 0 | 0 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry export import\n",
       "1 AERODEF  0      0     \n",
       "2 AERODEF  0      0     \n",
       "3 AERODEF  0      0     \n",
       "4 AERODEF  0      0     \n",
       "5 AERODEF  0      0     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(symbol_list = list(symbol)), keyby = .(industry, date)\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        for (i in 2:.N) {\n",
    "            intec <- intersect(symbol_list[[i]], symbol_list[[i-1]]) %>% length()\n",
    "            len1 <- length(symbol_list[[i]])\n",
    "            len2 <- length(symbol_list[[i-1]])\n",
    "            l[[i]] <- list(export = len2-intec, import = len1-intec)\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }, keyby = industry\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 将股票代码`symbol`按照日期`date`和行业`industry`，进行list化，每个行业每天的股票都将其变为一个单独的list`symbol_list`，这么做有利于之后进行向量之间取交集的操作。\n",
    "- line 2 对每个行业进行分组循环操作，遍历所有日期，首先对每一期和上一期的`symbol_list`取交集：`intersect(...)`，并得出这个交集向量的长度`intec`；接下来分别得出每一期和上一期`symbol_list`的长度：`len1`和`len2`；最后，用`len2-intec`得到上一期编出该行业的股票数量`export`，`len1-intec`得到这一期编入该行业的股票数量`import`，将这个有两个变量的list用`rbindlist`纵向拼接。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 57. 每天每个行业内股票收益率的标准差是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题关键在于分组计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>industry</th><th scope=col>date</th><th scope=col>ret_sd</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>AERODEF</td><td>20120104</td><td>0.02836880</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120105</td><td>0.02902021</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120106</td><td>0.03193141</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120109</td><td>0.01377493</td></tr>\n",
       "\t<tr><td>AERODEF</td><td>20120110</td><td>0.01417229</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 3\n",
       "\\begin{tabular}{lll}\n",
       " industry & date & ret\\_sd\\\\\n",
       " <chr> & <int> & <dbl>\\\\\n",
       "\\hline\n",
       "\t AERODEF & 20120104 & 0.02836880\\\\\n",
       "\t AERODEF & 20120105 & 0.02902021\\\\\n",
       "\t AERODEF & 20120106 & 0.03193141\\\\\n",
       "\t AERODEF & 20120109 & 0.01377493\\\\\n",
       "\t AERODEF & 20120110 & 0.01417229\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 3\n",
       "\n",
       "| industry &lt;chr&gt; | date &lt;int&gt; | ret_sd &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| AERODEF | 20120104 | 0.02836880 |\n",
       "| AERODEF | 20120105 | 0.02902021 |\n",
       "| AERODEF | 20120106 | 0.03193141 |\n",
       "| AERODEF | 20120109 | 0.01377493 |\n",
       "| AERODEF | 20120110 | 0.01417229 |\n",
       "\n"
      ],
      "text/plain": [
       "  industry date     ret_sd    \n",
       "1 AERODEF  20120104 0.02836880\n",
       "2 AERODEF  20120105 0.02902021\n",
       "3 AERODEF  20120106 0.03193141\n",
       "4 AERODEF  20120109 0.01377493\n",
       "5 AERODEF  20120110 0.01417229"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1), keyby = .(symbol, industry, date)\n",
    "    ][, .(ret_sd = sd(stkcd_ret)), keyby = .(industry, date)\n",
    "    ][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每一只股票的日收益率`stkcd_ret`。\n",
    "- line 2 以行业`industry`和日期`date`分组计算每天每个行业内股票收益率的标准差`ret_sd`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 58. 每天每个行业内股票收益率的标准差的相关性如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题在`Ex-57`的基础上，将每天每个行业内股票收益率的标准差求相关系数矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A matrix: 5 × 5 of type dbl</caption>\n",
       "<thead>\n",
       "\t<tr><th></th><th scope=col>AERODEF</th><th scope=col>AIRLINE</th><th scope=col>AUTO</th><th scope=col>BANKS</th><th scope=col>BEV</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>AERODEF</th><td>1.00000000</td><td>0.11523031</td><td>0.33195634</td><td>0.08021524</td><td>0.02497865</td></tr>\n",
       "\t<tr><th scope=row>AIRLINE</th><td>0.11523031</td><td>1.00000000</td><td>0.22235548</td><td>0.02446198</td><td>0.02250268</td></tr>\n",
       "\t<tr><th scope=row>AUTO</th><td>0.33195634</td><td>0.22235548</td><td>1.00000000</td><td>0.07283962</td><td>0.26374233</td></tr>\n",
       "\t<tr><th scope=row>BANKS</th><td>0.08021524</td><td>0.02446198</td><td>0.07283962</td><td>1.00000000</td><td>0.06073737</td></tr>\n",
       "\t<tr><th scope=row>BEV</th><td>0.02497865</td><td>0.02250268</td><td>0.26374233</td><td>0.06073737</td><td>1.00000000</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A matrix: 5 × 5 of type dbl\n",
       "\\begin{tabular}{r|lllll}\n",
       "  & AERODEF & AIRLINE & AUTO & BANKS & BEV\\\\\n",
       "\\hline\n",
       "\tAERODEF & 1.00000000 & 0.11523031 & 0.33195634 & 0.08021524 & 0.02497865\\\\\n",
       "\tAIRLINE & 0.11523031 & 1.00000000 & 0.22235548 & 0.02446198 & 0.02250268\\\\\n",
       "\tAUTO & 0.33195634 & 0.22235548 & 1.00000000 & 0.07283962 & 0.26374233\\\\\n",
       "\tBANKS & 0.08021524 & 0.02446198 & 0.07283962 & 1.00000000 & 0.06073737\\\\\n",
       "\tBEV & 0.02497865 & 0.02250268 & 0.26374233 & 0.06073737 & 1.00000000\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A matrix: 5 × 5 of type dbl\n",
       "\n",
       "| <!--/--> | AERODEF | AIRLINE | AUTO | BANKS | BEV |\n",
       "|---|---|---|---|---|---|\n",
       "| AERODEF | 1.00000000 | 0.11523031 | 0.33195634 | 0.08021524 | 0.02497865 |\n",
       "| AIRLINE | 0.11523031 | 1.00000000 | 0.22235548 | 0.02446198 | 0.02250268 |\n",
       "| AUTO | 0.33195634 | 0.22235548 | 1.00000000 | 0.07283962 | 0.26374233 |\n",
       "| BANKS | 0.08021524 | 0.02446198 | 0.07283962 | 1.00000000 | 0.06073737 |\n",
       "| BEV | 0.02497865 | 0.02250268 | 0.26374233 | 0.06073737 | 1.00000000 |\n",
       "\n"
      ],
      "text/plain": [
       "        AERODEF    AIRLINE    AUTO       BANKS      BEV       \n",
       "AERODEF 1.00000000 0.11523031 0.33195634 0.08021524 0.02497865\n",
       "AIRLINE 0.11523031 1.00000000 0.22235548 0.02446198 0.02250268\n",
       "AUTO    0.33195634 0.22235548 1.00000000 0.07283962 0.26374233\n",
       "BANKS   0.08021524 0.02446198 0.07283962 1.00000000 0.06073737\n",
       "BEV     0.02497865 0.02250268 0.26374233 0.06073737 1.00000000"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1), keyby = .(symbol, industry, date)\n",
    "    ][, .(ret_sd = sd(stkcd_ret)), keyby = .(industry, date)\n",
    "    ][, dcast(.SD, date ~ industry, value.var = \"ret_sd\")\n",
    "    ][, cor(.SD[, -1])\n",
    "    ][1:5, 1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 和 line 2 可完全参考`Ex-57`。\n",
    "- line 3 保留日期`date`这个变量，而后将`industry`里的所有行业观测值，变为变量名，而后在这些变量名里填充各个行业每天的收益率标准差`ret_sd`。即`dcast(.SD, date ~ industry, value.var = \"ret_sd\")`。\n",
    "- line 4 求这些行业收益率标准差之间的相关系数矩阵`cor(.SD[, -1])`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 59. 每天计算出成交额的 z-score （减去均值除以标准差）, 该指标能解释下一天个股超额收益率的多少比例？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本题为个股超额收益率和z-score计算的综合应用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.000985359508317689"
      ],
      "text/latex": [
       "0.000985359508317689"
      ],
      "text/markdown": [
       "0.000985359508317689"
      ],
      "text/plain": [
       "[1] 0.0009853595"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, weight = capt/sum(capt), symbol, amount), keyby = date\n",
    "    ][, .(mkt_ret = sum(weight*stkcd_ret), stkcd_ret, symbol, amount), keyby = date\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ mkt_ret))[1], beta = coef(lm(stkcd_ret ~ mkt_ret))[2], mkt_ret, stkcd_ret, date, amount), keyby = symbol\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * mkt_ret, amount), keyby = .(date, symbol)\n",
    "    ][, .(abnr_ret, zscore = (amount - mean(amount))/ sd(amount), symbol), keyby = date\n",
    "    ][, .(abnr_lead_ret = shift(abnr_ret, n = 1L, type = \"lead\"), zscore, symbol), keyby = symbol\n",
    "    ][, summary(lm(abnr_lead_ret ~ zscore))$r.squared]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 ~ line 4 为个股超额收益率的计算，可完全参考`Ex-30`，此处不做赘述。\n",
    "- line 5 计算个股每天成交额的的z-score`zscore`。\n",
    "- line 6 将所有个股超额收益率`abnr_ret`提前一期，生成`abnr_lead_ret`。\n",
    "- line 7 最后用`zscore`对`abnr_lead_ret`进行回归，提取回归的`r-squared`，即为解释比例."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 60. 每个股票的收益率和300、500指数收益率可以回归出一个截距项和2个beta，这两个beta的分布如何？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于将每一只股票对300和500指数进行分组回归并取出两个beta值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 2 × 3</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>tag</th><th scope=col>sample_mean</th><th scope=col>sample_sd</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;dbl&gt;</th><th scope=col>&lt;dbl&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>ind_w300_ret</td><td>-0.1583057</td><td>0.9317869</td></tr>\n",
       "\t<tr><td>ind_w500_ret</td><td> 1.1264276</td><td>0.8311509</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 2 × 3\n",
       "\\begin{tabular}{lll}\n",
       " tag & sample\\_mean & sample\\_sd\\\\\n",
       " <chr> & <dbl> & <dbl>\\\\\n",
       "\\hline\n",
       "\t ind\\_w300\\_ret & -0.1583057 & 0.9317869\\\\\n",
       "\t ind\\_w500\\_ret &  1.1264276 & 0.8311509\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 2 × 3\n",
       "\n",
       "| tag &lt;chr&gt; | sample_mean &lt;dbl&gt; | sample_sd &lt;dbl&gt; |\n",
       "|---|---|---|\n",
       "| ind_w300_ret | -0.1583057 | 0.9317869 |\n",
       "| ind_w500_ret |  1.1264276 | 0.8311509 |\n",
       "\n"
      ],
      "text/plain": [
       "  tag          sample_mean sample_sd\n",
       "1 ind_w300_ret -0.1583057  0.9317869\n",
       "2 ind_w500_ret  1.1264276  0.8311509"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, ind_w300_ret = sum((close/pre_close - 1) * index_w300), ind_w500_ret = sum((close/pre_close - 1) * index_w500), symbol), keyby = date\n",
    "    ][, .(coef_beta = coef(lm(stkcd_ret ~ ind_w300_ret + ind_w500_ret))[-1], tag = c(\"ind_w300_ret\", \"ind_w500_ret\")), keyby = symbol\n",
    "    ][!is.na(coef_beta), .(sample_mean = mean(coef_beta), sample_sd = sd(coef_beta)), keyby = tag]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 分别计算每只股票、300指数和500指数每天的收益率。\n",
    "- line 2 将300指数和500指数分别对每只股票做回归，并提取出每只股票的两个beta系数，并生成一个变量标记出beta值属于300还是500指数。\n",
    "- line 3 计算这两个beta值的均值和方差。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 61. 每天开盘后到最高价涨幅最大的100只股票同样也是全天(昨收到今收)涨幅最大的100只股票的比例是多少?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题关键在于计算每只股票开盘价到最高价的涨幅和每个交易日的涨幅，而后进行降序排序，分别选出前100只股票。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "0"
      ],
      "text/latex": [
       "0"
      ],
      "text/markdown": [
       "0"
      ],
      "text/plain": [
       "[1] 0"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(high_open = high/open - 1), by = .(symbol, date)\n",
    "    ][order(date, -high_open), .(ho_symbol = list(symbol[1:100])), \n",
    "      keyby = date\n",
    "    ][data[, .(clo_pre = close/pre_close - 1), by = .(symbol, date)    \n",
    "        ][order(date, -clo_pre), .(cp_symbol = list(symbol[1:100])), \n",
    "          keyby = date], on = .(date)\n",
    "    ][, .(len = (intersect(unlist(cp_symbol), unlist(ho_symbol)) %>% length)), keyby = date\n",
    "    ][, sum(len == 100)/.N]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 计算每只股票开盘价到最高价的涨幅`high_open`。\n",
    "- line 2 按日期`date`升序并按照`high_open`降序排列，选择出前100的股票，并分日期进行list化`ho_symbol`，这里进行list化是为了进行merge的时候，对key的选择的唯一性问题的方便，因为这样两个表就可以有一个unique key名为`date`。\n",
    "- line 3 重复line 1 和 line 2的操作统计出每天涨幅前100的股票`clo_open`，并和之前的表按照日期`date`进行合并。\n",
    "- line 4 将所创建的两个股票代码的list先向量化，而后求交集`intersect(unlist(cp_symbol), unlist(ho_symbol))`，并计算这个交集向量的长度`len`。\n",
    "- line 5 计算每个日期下面符合两个条件的比例，首先判断`len == 100`，再将所有符合条件的element加起来，再除以整个行数，就可以得出比例。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 62. 每天计算最近三天每天对市场的超额收益率都排进当天前100的股票有哪些?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题为超额收益率和交集问题的结合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning message in rbindlist(l):\n",
      "“Column 1 ['symbol'] of item 10 is length 0. This (and 12 others like it) has been filled with NA (NULL for list columns) to make each item uniform.”\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>date</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>600462.SH</td><td>20120109</td></tr>\n",
       "\t<tr><td>600228.SH</td><td>20120109</td></tr>\n",
       "\t<tr><td>000791.SZ</td><td>20120110</td></tr>\n",
       "\t<tr><td>600792.SH</td><td>20120111</td></tr>\n",
       "\t<tr><td>600740.SH</td><td>20120111</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & date\\\\\n",
       " <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 600462.SH & 20120109\\\\\n",
       "\t 600228.SH & 20120109\\\\\n",
       "\t 000791.SZ & 20120110\\\\\n",
       "\t 600792.SH & 20120111\\\\\n",
       "\t 600740.SH & 20120111\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | date &lt;int&gt; |\n",
       "|---|---|\n",
       "| 600462.SH | 20120109 |\n",
       "| 600228.SH | 20120109 |\n",
       "| 000791.SZ | 20120110 |\n",
       "| 600792.SH | 20120111 |\n",
       "| 600740.SH | 20120111 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    date    \n",
       "1 600462.SH 20120109\n",
       "2 600228.SH 20120109\n",
       "3 000791.SZ 20120110\n",
       "4 600792.SH 20120111\n",
       "5 600740.SH 20120111"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, weight = capt/sum(capt), symbol), keyby = date\n",
    "    ][, .(mkt_ret = sum(weight*stkcd_ret), stkcd_ret, symbol), keyby = date\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ mkt_ret))[1], beta = coef(lm(stkcd_ret ~ mkt_ret))[2], mkt_ret, stkcd_ret, date), keyby = symbol\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * mkt_ret), keyby = .(date, symbol)\n",
    "    ][order(date, -abnr_ret), .(symbol = list(symbol[1:100])), keyby = date\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        for (t in 4:.N) {\n",
    "            l[[t]] <- list(symbol = Reduce(intersect, list(symbol[[t]], symbol[[t-1]], symbol[[t-2]])), date = date[[t]])\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }][1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- line 1 ~ line 4 为股票超额收益率的计算过程，可完全参考`Ex-30`在此不作赘述。\n",
    "- line 5 以日期`date`升序和超额收益率`abnr_ret`降序排列，选择出每天超额收益率都在前100的股票，并list化。\n",
    "- line 6 由于需要选择出最近三天超额收益率都进前一百的股票，那么对整体进行循环，对`t`、`t-1`和`t-2`三天超额收益率都进入的100股票进行交集的计算，合并上日期`date`，整体进行输出。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 63. 每天计算最近三天每天对行业的超额收益率都排进当天行业前30%的股票有哪些?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此题为`Ex-62`的变形，可完全参考，此处不做赘述。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<caption>A data.table: 5 × 2</caption>\n",
       "<thead>\n",
       "\t<tr><th scope=col>symbol</th><th scope=col>date</th></tr>\n",
       "\t<tr><th scope=col>&lt;chr&gt;</th><th scope=col>&lt;int&gt;</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "\t<tr><td>000009.SZ</td><td>20120109</td></tr>\n",
       "\t<tr><td>000027.SZ</td><td>20120109</td></tr>\n",
       "\t<tr><td>000039.SZ</td><td>20120109</td></tr>\n",
       "\t<tr><td>000045.SZ</td><td>20120109</td></tr>\n",
       "\t<tr><td>000411.SZ</td><td>20120109</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "A data.table: 5 × 2\n",
       "\\begin{tabular}{ll}\n",
       " symbol & date\\\\\n",
       " <chr> & <int>\\\\\n",
       "\\hline\n",
       "\t 000009.SZ & 20120109\\\\\n",
       "\t 000027.SZ & 20120109\\\\\n",
       "\t 000039.SZ & 20120109\\\\\n",
       "\t 000045.SZ & 20120109\\\\\n",
       "\t 000411.SZ & 20120109\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "A data.table: 5 × 2\n",
       "\n",
       "| symbol &lt;chr&gt; | date &lt;int&gt; |\n",
       "|---|---|\n",
       "| 000009.SZ | 20120109 |\n",
       "| 000027.SZ | 20120109 |\n",
       "| 000039.SZ | 20120109 |\n",
       "| 000045.SZ | 20120109 |\n",
       "| 000411.SZ | 20120109 |\n",
       "\n"
      ],
      "text/plain": [
       "  symbol    date    \n",
       "1 000009.SZ 20120109\n",
       "2 000027.SZ 20120109\n",
       "3 000039.SZ 20120109\n",
       "4 000045.SZ 20120109\n",
       "5 000411.SZ 20120109"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data[, .(stkcd_ret = close/pre_close - 1, symbol, weight = capt/sum(capt)), keyby = .(industry,date)\n",
    "    ][, .(ind_ret = sum(weight*stkcd_ret), stkcd_ret, symbol), keyby = .(industry, date)\n",
    "    ][, .(alpha = coef(lm(stkcd_ret ~ ind_ret))[1], beta = coef(lm(stkcd_ret ~ ind_ret))[2], ind_ret, stkcd_ret, symbol, date), keyby = .(symbol, industry)\n",
    "    ][, .(abnr_ret = stkcd_ret - alpha - beta * ind_ret), keyby = .(date, symbol)\n",
    "    ][order(date, -abnr_ret) & abnr_ret > quantile(abnr_ret, 0.7), .(symbol = list(symbol)), keyby = date\n",
    "    ][, {\n",
    "        l <- list()\n",
    "        for (t in 4:.N) {\n",
    "            l[[t]] <- list(symbol = Reduce(intersect, list(symbol[[t]], symbol[[t-1]], symbol[[t-2]])), date = date[[t]])\n",
    "        }\n",
    "        rbindlist(l)\n",
    "    }][1:5]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R-3.6.3",
   "language": "R",
   "name": "ir36"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.6.3"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
